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ABSTRACT 


There  are  many  reasons  for  empirically  testing 
hypotheses  about  the  effects  of  various  factors  on  the 
psychological  complexity  of  computer  programs.  (By 
’’psychological  complexity”  we  mean  the  intrinsic  property  of 
urograms  that  affects  their  under standability  and 
maintainability.)  This  thesis  develops  a  methodology  for 
such  experimentation,  and  discusses  the  results  of  ten 
experiments  involving  the  following  factors:  use  of 
comments,  control  flow,  paragraphing,  choice  of  variaola 
names,  and  locality  of  data  references. 

The  methodology  consists  of  a  factorial  design  with 
each  subject  being  given  a  program  in  which  various  factors 
occur  at  various  levels.  Subjects  are  asked  to  read,  study 
in  detail,  and  modify  the  programs.  Objective  and 
subjective  measures  are  taken  of  the  subjects'  performance 
throughout  the  experiment.  Analysis  of  variance  is  used  to 
analyze  the  resulting  data.  The  individual  components  of 
the  meth odologv  are  d escribed  in  detail  and  the  capabilities 
and  limitations  of  the  methodology  as  a  whole  are  discussed. 

The  experiments  produced  statistically  significant 
results  involving  all  ^ive  factors  studied.  In  general,  the 
results  confirmed  our  intuitive  impressions,  although  we 
cannot  draw  definite  conclusions  about  any  of  the  factors 
because  of  the  limited  number  of  experiments  performed  and 
the  similarity  of  the  background  of  the  subjects  used. 

Cur  main  conclusion  is  that  our  methodology  meets  the 
requirements  of  feasibility  and  credibility.  We  feel  that 


it  will  provide  a  substantial  basis  for  future  research  in 
this  area  and  we  suggest  possible  directions  for  such 


research. 
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Chapter  1 
INTRODUCTION 


1 , 1  Goals  of  the  Thesis 

1.1.1  What  is  Fs vch ol2£ical  Complexity? 

The  major  goal  of  this  thesis  is  the  development  of  a 
methodology  for  studying  the  psychological  complexity  of 
computer  programs.  Some  computer  programs  are  more 
difficult  to  understand  and  maintain  than  others.  3y 
"psychological  complexity"  we  mean  all  those  aspects  of 
programs  that  contribute  to  this  difficulty.  By  complexity 
factor  we  mean  any  one  aspect.  Some  of  a  program's 
complexity  may  be  inherent  in  the  problem  to  be  solved,  and 
some  may  come  from  the  algorithm  selected,  but  a  major  part 
will  be  determined  by  the  "style"  of  programming  that  is 
used  [Kernighan  and  Plauger  1974].  It  is  this  latter  kind 
of  complexity  that  can  be  reduced  most  easily,  when  we 
understand  the  factors  contributing  to  it.  In  developing 
our  methodology,  we  will  therefore  be  concerned  primarily 
with  this  kind  of  complexity. 

This  thesis  is  based  on  four  assumptions  about  the 
psychological  complexity  of  computer  programs.  The  first 
assumption  is  that  psychological  complexity  is  a  real, 
underlying  attribute  of  programs  that  is  often  ignored 
because  it  is  not  easy  to  define  or  describe,  let  alone 
measure.  Harlan  Mills  emphasizes  this  point  [Mills  1972]: 
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"In  computer  programming 

today 

we  do 

not  yet 

know 

that  'complexity  has  weight'. 

Since 

it  is 

not  ea 

s  ily 

measured  or  described,  ... 

we 

often 

ignore 

the 

complexity  of  a  planned  program  or 

subp 

rogram. 

But 

when  this  complexity  exceeds  certain  unknown  limits, 
frustration  ensues.  Computer  programs  capsize  under 
their  own  logical  weight,  or  become  so  crippled  that 
maintenance  is  precarious  and  modification  is 
impossible. " 


Our  second  as 

sum  pt ion  is 

that 

this  complexity 

consistently  affects 

one's  ability 

to 

understand  and 

maintain  programs. 

(By  maintenance 

,  we 

mean  both  fixing 

errors  and  modifying 

the  function 

of 

programs.)  The 

difficulty  that  a  p 

rogrammer  has 

with 

understanding  and 

maintaining  a  program. 

however,  is  aff 

ected 

not  only  by  the 

intrinsic  complexity 

of  the  program. 

but 

also  by  his  skill 

and  the  extent  to  which  he  has  studied  the  program. 

Our  third  assumption  is  that  we  can  indirectly  estimate 
this  underlying  complexity  by  using  particular  measures  of 
under standabi lity  and  maintainability. 

Our  final  assumption  is  that  psychological  complexity 
is  composed  of  various  factors  whose  effects  on 
under standability  and  maintainability  can  be  studied  more  or 
less  independently  and  that  things  that  affect  the 
complexity  of  small  programs  also  affect  the  complexity  of 
large  programs.  The  results  obtained  from  studying  these 
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factors  can  then  be  used  to  guide  the  construction  of  less 
complex  programs. 


1.1.2  Why;  Study  Complexity? 

If  psychological  complexity  is  a  real  attribute  of 
urograms,  there  are  many  reasons  for  studying  it.  f?or  many 
years,  programming  was  considered  to  be  a  machine- oriented 
activity,  and  this  augmented  interest  in  computational 
complexity  [Borodin  1973].  Unfortunately,  little  attention 
w as  given  to  the  human  aspect.  It  was  pointed  out  by 
Dijkstra  [1965],  and  more  recently  by  Weinberg  [1971],  that 
programming  is  in  fact  a  human  activity,  that  it  is  people 
who  write  programs  and  people  who  must  read  and  modify 
programs  written  by  other  people.  The  failure  to  recognize 
explicitly  and  deal  with  this  for  so  many  years  has  caused  a 
number  of  problems. 


Tn  the  early  days  of  programming,  a  programmer's  goal 
was  often  to  produce  the  most  efficient  program  possible. 
Coding  tricks  that  reduced  the  size  or  execution  time  of  a 
program  were  highly  regarded.  Very  little  attention  was 
given  to  how  easy  it  was  for  someone  other  than  the  author 
of  a  program  to  be  able  to  read  or  modify  the  program.  This 
led  to  the  production  of  a  large  number  of  complex, 
inscrutable  programs. 


As  large  systems  developed  and  programmer  turnover  on 
projects  increased. 


this  situation 


was 


found 


to 


be 
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unsat isf actory .  For  example,  when  a  change  had  to  be  made 
ir.  a  particular  program  and  the  original  programmer  was  no 
longer  available,  it  was  often  easier  to  rewrite  the  entire 
croaram  than  to  attempt  to  make  the  modification.  It  was 
also  found  that  complexity  of  large  systems  was  increased 
when  a  programmer  working  on  a  particular  piece  of  a  system 
used  knowledge  about  the  other  portions  of  the  system  in 
order  to  gain  efficiency.  This  often  led  to  the  inability 
to  change  a  piece  of  the  system  without  having  to  make 
charges,  often  extensive,  to  other,  seemingly  unrelated 
pieces  of  the  system. 

?^ore  recently  [Naur  and  Pandell  1968,  Buxton  and 
Pandell  1969],  attention  has  been  focussed  on  the  fact  that 
complexity  of  programs  must  be  drastically  reduced  to  aid  in 
their  understanding  and  maintenance.  In  order  to  reduce  the 
complexity  of  programs,  many  ideas  and  techniques  have  been 
expounded.  These  include  documentation  standards  for 
programmers,  the  use  of  high  level  languages  for  system 
implementation,  and  most  notably,  the  idea  of  structured 
programming.  Much  literature  has  appeared  under  the  banner 
of  structured  programming  [Dijkstra  1971a,  Dijkstra  1971b, 
Dijkstra  1972,  Mills  1972,  Naur  1972,  Parnas  1971,  '/firth 
ll^l].  Since  it  is  such  a  general  concept,  many  techniques, 
often  contradictory  ones,  have  been  presented  that  claim  to 
be  examples  of  structured  programming.  Too  often,  however, 
no  evidence  is  given  that  a  technique  is  indeed  useful. 


W e  contend  that 


in  order  to  show  the  usefulness  of  a 


programming  technique  or  language  construct,  empirical 
evidence  from  programmers'  performance  should  be  obtained, 
we  have  conducted  controlled  experimental  studies  to  measure 
some  factors  that  we  feel  might  make  programs 
psychologically  complex.  We  hope  the  results  of  such 
studies  will  provide  data  that  will  aid  both  in  language 
design  and  in  writing  programs  that  are  easy  to  understand 
and  maintain. 

In  section  4.1,  we  present  a  list  of  factors  that  we 
originally  thought  might  contribute  to  the  complexity  of 
programs.  Our  original  goal  when  we  began  this  research  was 
to  get  all  the  answers,  that  is,  to  determine  the  effects  of 
all  the  complexity  factors  on  program  un derstandab ility  and 
maintainability.  It  soon  became  apparent  that  this  attempt 
was  premature.  Conducting  experiments  with  programmers  was 
such  a  new  area  of  research  that  no  standard  methodologies 
existed  that  could  be  directly  applied.  In  addition,  we 
only  had  a  small  number  of  subjects  available,  which  greatly 
limited  the  number  of  experiments  we  could  conduct. 

Consequently,  our  major  goal  became  the  establishment 
of  a  suitable  methodology  for  conducting  experiments  that 
could  be  used  with  a  larger  number  of  subjects  for  measuring 
program  complexity.  We  concentrated  primarily  on  a 
methodology  for  measuring  understanding  and  modifiability 
since  some  work  has  already  been  done  by  others  on  measuring 
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the  ability  to  fix  errors  in  programs.  This  previous  work 
will  be  described  in  the  next  chapter. 

Although  the  major  result  of  the  thesis  is  the 
presentation  of  our  methodology,  we  also  obtained  results 
about  specific  complexity  factors.  These  results,  although 
secondary  to  the  establishment  of  the  methodology,  are 
themselves  interesting. 

1.1.3  Requirement s  for  Our  Methodology 

There  are  two  categories  of  requirements  that  any 
experimental  methodology  must  meet,  feasibility  and 
credibility.  In  this  section,  we  will  explain  what  we  mean 
bv  feasibility  and  credibility  of  our  methodology  for 
studying  the  psychological  complexity  of  computer  programs. 

For  such  a  methodology  to  be  feasible,  it  must  be  easy 
to  prepare  and  administer.  Although  a  background  in 
experimental  psychology  and  statistics  would  prove  helpful, 
it  must  be  possible  f or  a  computer  scientist  with  minimal 
training  in  psychology  and  only  a  reading  knowledge  of 
statistics  to  prepare  and  conduct  an  experiment  and  also 
interpret  the  results. 

Such  a  methodology  must  also  work  with  the  available 
subjects.  In  particular,  students  and  programmer  trainees 
are  the  most  likely  candidates,  although  the  methodology 
should  work  with  experienced  programmers  as  well.  In 
addition,  the  methodology  must  fit  into  a  reasonable  time 
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frame  because  of  the  cost  and/or  availability  of  subjects. 
It  also  must  not  require  excessive  time  of  the  experimenter, 
which,  for  example,  would  rule  out  testing  subjects 
individually.  In  addition,  the  experiment  must  not  require 
elaborate  equipment  that  would  be  difficult  or  expensive  to 
obtain.  Finally,  for  the  methodology  to  be  feasible,  it 
must  work  for  the  kinds  of  factors  that  we  are  interested  in 
studying. 

The  methodology  must  also  obtain  results  that  are 
credible.  One  important  aspect  of  the  credibility  of  a 
methodology  is  the  inherent  "reasonableness"  of  the 
procedures.  This  includes  the  proper  choice  and  combination 
of  the  components  of  the  methodology  and  in  particular  the 
choice  of  measures  used.  In  addition,  the  measures  chosen 
must  be  easily  quantified. 


Another  aspect  of  credibility  involves  th 
of  results.  In  order  for  +he  results  of  an  exp 
convincing,  they  must  be  repeatable  in  othe 
conducted  with  different  subjects  under 
experimental  conditions.  Also,  within  a  give 
the  results  obtained  on  various  measures  should 
agree  in  direction,  although  the  magnitude 
vary  from  measure  to  measure. 


e  consist 
eriment  t 
r  experim 
diffe 
n  experim 
,  in  gens 
of  effect 


e 

o 

0 

r 


ncy 

be 

nts 

ent 

nt, 

al, 

may 


Finally,  the  results  obtained  must  be  "reasonable". 
Tnis  means  that  they  should  be  explainable  either  on  the 
basis  of  work  done  by  others  or  on  the  basis  of  intuitive 
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argumer.ts.  We  do  not  expect  all  the  results  obtained  to 
confirm  intuitive  impressions.  However,  if  most  of  the 
results  obtaired  were  not  reasonable,  this  would  reduce  the 
credibility  of  the  methodology. 


1.2  Outline  of  the  Thesis 

Chapter  2  surv 
presented  in  the  lite 
the  need  for  measu 
presenting  a  diverse 
section  surveys  act 
for  such  studies)  con 
them  with  our  researc 

Chapter  3  then 
the  effects  of  variou 
After  presenting  an  o 
components  are  descri 
as  a  whole,  includi 
discussed. 

Chapter  4  presents 


factors  and 

discusses 

particular  f 

actors  stud 

control  flow. 

par agraphin 

locality  of 

data  refer 

results  were 

obtained  abo 

list 

of  progra 

m  complexity 

e  suits 

obtained 

about  the 

wh  ich 

were  use 

of  comments. 

ice  of 

variable 

names,  and 
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istically 

significant 

h  of  them. 


eys  the  background  for  the  thesis,  as 
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ring  the  complex it  y  of  programs  by 
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Chapter  5  summarizes  the  principal  results  of  rhe 
thesis.  In  addition,  suggestions  for  further  research  in 
psychological  complexity  of  computer  programs  are  presented. 
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Chapter  2 

BACKGROUND  AND  LITERATURE  SURVEY 
i  .  1  ThH  for  leasurinc[  the  Complexity  of  Programs 

Even  before  studies  were  started  to  measure  the 
complexity  of  programs,  the  need  for  such  measures  was 
acknowledged  in  Ahe  literature.  This  section  contains 
typical  examples  from  the  literature  pointing  out  this  need. 
The  examples  indicate  that  many  otherwise  unrelated  studres 
have  in  common  the  suggestion  of  measuring  program 
complexity. 

Dijkstra  is  undoubtedly  the  leading  proponent  of 
structured  programming.  He  has  designed  and  implemented  an 
operating  system,  "THI"  [Dijkstra  1968b],  which  was 
structured  so  as  to  or  able  him  to  convince  himself  as  to  its 
Logical  correctness  prior  to  its  running  on  the  computer. 
qe  describes  many  of  the  techniques  and  insights  gained  cn 
-his  project  ir.  r Dijkstra  1972  ].  Dijkstra  refers  to 
programming  as.  "the  ait  cf  organizing  complexity".  He  gives 
examples  of  his  technique  of  stop-wise  program  composition 
ir.  which  programs  are  composed  in  levels,  each  level 
refining  concepts  that  were  used  in  higher  levels.  Each 
level  defines  a  new  virtual  machine.  Dijkstra  visualizes  a 
program  as  a  necklace  with  various  levels  considered  to  be 
"pearls".  He  admits  +  hai  his  particular  structuring  of  a 
problem  may  not  necessarily  be  the  best  possible  structuring 
and  points  out  4-he  need  for  "a  yardstick  by  which  to  measure 
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the  usefulness  of  a  proposed  structuring”.  Dijkstra  paints 
out  an  interesting  quantitative  aspect  to  his  ’’pearl”  model. 
"Along  the  necklace  we  can  indicate  for  each  concept  its 
range  of  validity:  of  course  they  overlap  and  we  can  view 


t  hem 

as  the 
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consisting  of  more  and  longer  individual  threads  than  a 
veil- structured  program.  An  attempt  to  implement  Dijksrra's 
ideas  to  allow  a  programmer  to  compose  his  programs  in 
levels  has  been  made  in  the  PEAHL  system  [Snowden  1972]. 

Parnas  is  also  concerned  with  the  structure  of  systems. 
In  [Parnas  1968],  he  concludes  that  the  term  "process"  which 
is  often  used  in  structural  system  description  cannot  be 
given  a  satisfactory,  precise  definition.  As  such,  he 
introduces  the  concept  " processi vity"  and  gives  examples  of 
sets  of  events  with  high  and  low  processi vity .  He  mentions 
the  idea  of  a  function  for  processivity  with  which  to 
measure  a  proposed  decomposition  of  a  system.  He  concludes, 
however,  that  many  different  functions  are  possible, 
depending  on  the  uses  to  which  they  will  be  put. 


P lexander 
information  trans 
[Alexander  1964  ] 
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design.  The  vertices  of  the  graph  represent  "mis 

variables",  or  negative  design  criteria,  and  the  ed 
represent  links  between  the  misfit  variables.  He  t 
successively  parti ti on s  the  graph  by  minimizing  a  particu 
measure  of  the  information  transfer  among  the  subsets.  T 
provides  a  hierarchical  set  of  sub-problems  which  can 
solved  and  synthesized  into  a  complete  desi 

Unfortunately,  the.  main  part  of  the  design  problem  is  1 
unsolved.  Alexander  does  not  provide  a  procedure 
determining  the  misfit  variables  for  a  particular  prob 
nor  the  relationships  between  them. 
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the  maintenance  effor-1-.  to  the  consent,  structure,  and 
complexity  of  the  programs  contained  within  a  system.  This 
micro-model  complexity  appears  to  be  the  same  as  what  we 
have  been  calling  psychological  complexity. 

Another  look  at  the  maintenance  effort  has  been  made  by 
McKeeman.  In  a  talk  given  on  fracture  planes  in  compilers 
r  McKee  mar:  1972c],  he  made  several  conjectures.  He 
conjectured  that  the  average  number  of  changes  in  a  module 
du-  directly  to  a  change  in  another  module  is  a  function  of 
the  assumptions  which  the  modules  make  about  each  other.  He 
also  conjectured  that  the  average  cost  of  a  change  in  a 
module  is  a  function  of  the  number  of  paths  in  the  module 
affected  by  the  change.  Although  the  conjectures  seem 
plausible,  they  do  need  to  be  validated  in  some  way. 


2*  2  experimental  Studies 

Sackman  is  probably  one  of  the  earliest  proponents  of 
the  idea  of  conducting  experiments  with  programmers  [Sackman 
1968].  He  not  only  points  out  the  lack  of  such  studies,  but 
also  emphasizes  their  importance.  It  is  Weinberg,  however, 
who  has  widely  spread  the  idea  in  his  book  [Weinberg  1971], 

Weinberg  considers  programming  as  a  human  activity 
rather  than  a  machine  activity  and  points  out  that  methods 
from  the  behavioural  sciences  can  be  used  to  study  it.  He 
claims  that  because  programming  is 


such 


complicated 
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behaviour,  results  from  the  behavioural  sciences  cannot  be 
directly  applied  although  we  can  learn  from  their 
a  pproaches. 


Weinberg  advocates  controlled  experimentation  with 
programmers  but  points  out  many  of  the  pitfalls.  These 
include  the  elimination  of  interesting  results  due  to  the 
constrained  nature  of  the  experimental  situation,  the 
biasing  of  the  results  due  to  the  "experience1'  of  the 
subjects,  and,  finally,  the  large  cost  involved  in  obtaining 
subjects.  Although  much  of  his  book  contains  anecdotes  and 
interesting  personal  experiences,  it  also  contains  the 
results  of  various  experiments  he  has  performed.  Weinberg 
considers  programming  both  as  a  group  activity  and  as  an 
individual  activity.  As  a  group  activity,  he  considers  such 
aspects  as  egoless  programming,  team  organization  and 
leadership,  and  the  goals  set  for  teams.  As  an  individual 
activity,  Weinberg  considers  the  goals  set  for  programmers, 
and  the  personality  factors,  intelligence,  motivation, 
-raining,  and  experience  of  the  programmers.  He  also 
considers  the  effects  of  the  tools  available  to  programmers. 
It  is  clear  after  reading  the  book  that  experimentation  with 
programmers  can  yield  important  and  often  quite  surprising 
results. 


In 

study  pe 
required 


[Weinberg  1972],  Weinberg  reports  the  results  of  a 
rformed  with  five  programming  teams.  The  teams  were 
to  write  a  program  to  solve  the  same  problem,  but 
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each  team  was  given  a  different  objective.  The  five 
objectives  were  minimum  core,  output  clarity,  program 
clarity,  minimum  statements,  and  minimum  hours.  The  teams 
were  ranked  on  how  well  they  met  each  of  the  five 
objectives.  In  addition  to  the  fact  that  each  team  ranked 
first  in  meeting  its  primary  objective,  other  interesting 
results  appeared.  Certain  objectives  appear  to  be  in 
conflict  with  one  another.  For  example,  minimum  statements 
interferes  with  output  clarity,  minimum  core  interferes  with 
output  clarity,  and  promptness  (minimum  hours)  conflicts 
with  all  four  other  objectives. 


At  the  IBM  T.  J.  Watson  Research  Center,  several 
experimental  studies  of  the  programming  process  have  been 
conducted.  One  group  of  studies  has  been  concerned  with 
studying  errors  in  programs  and  the  debugging  process  [Boies 
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studies.  In  this  study  actual  program  submissions  were 
automatically  checked  for  syntactic  errors.  They  discovered 
that  about  five-sixths  of  all  the  programs  submitted, 
including  "first  submissions",  were  syntactically  correct. 
As  a  result  of  this,  the  next  two  studies  [Gould  and 
Dror.gowski  1972,  Gould  1973]  dealt  only  with  conceptual 
e  rror s. 
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These  studies  were  controlled  experiments  in  which 
several  factors  were  varied.  Programmers  were  asked  to  find 
conceptual  errors  in  syntactically  correct  programs.  Three 
classes  of  errors  and  five  different  levels  of  debugging 
aids  were  studied.  One  major  result  of  these  studies  was 
that  programmers  who  were  given  only  the  listing  were  able 
to  debug  almost  as  efficiently  as  programmers  who  were  given 
other  debugging  aids.  The  other  major  result  was  that 
programmers  who  were  told  the  exact:  line  that  contained  the 
error  and  were  asked  to  repair  it  took  about  half  as  much 
time  as  programmers  who  were  required  to  locate  the  srror 
but  not  repair  it.  They  claim  that  this  indicates  an  upper 
bound  for  debugging  since  telling  a  programmer  the  line  in 
which  an  error  occurs  is  just  about  the  ultimate  debugging 
a  id. 
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Another  type  of  experimental  study  being  conducts!  at 
IBM  Yorktown  Heights  is  concerned  with  investigation  of  the 
programming  process  itself  using  "naive”  subjects,  i.e., 
subjects  with  no  training  in  programming.  In  the  first 
experiment  in  this  study  [Miller  1973],  subjects  were  asked 
to  program  simple  problems  by  selecting  commands  from  a 
laboratory  programming  language.  Two  factors  were  varied  in 
the  problem  specification.  The  first  factor  dealt  with  the 
type  of  conditions  the  program  would  have  to  test  for, 
either  "and"  or  "or"  conditions.  The  second  factor  dealt 
with  whether  the  conditions  to  be  tested  for  were  expressed 
affirmatively  or  negatively.  The  results  indicate  that  "or" 
conditions  were  much  harder  for  the  subjects  than  "and" 
conditions  and  that  negatively  expressed  conditions  were 
more  difficult  than  affirmatively  expressed  ones.  In 
addition,  the  results  were  linked  to  the  specific  task  of 
specifying  a  procedure  rather  than  to  conceptual 
difficulties. 

A  controlled  experiment  using  "naive"  subjects  has  also 
been  conducted  at  Sheffield  University  [ Sime  1973],  The 
purpose  of  this  study  was  to  test  the  feasibility  of  using 
micro-languages  to  isolate  language  constructs  about  which 
experimental  data  is  desired.  In  this  particular 
experiment,  two  micro-languages  were  used,  one  containing  a 
GOTO  construct  and  the  other  containing  an  IE_THEN_ELSE 
construct.  The  micro-language  containing  the  IF_THEN_ELSE 
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const  ruct  was  superior  in  minimizing  both  the  time  taken  to 
solve  problems  and  in  the  number  of  errors  made. 

?■_  somewhat  different  approach  to  studying  the  human 
aspects  of  programming  has  been  proposed  [Brooks  1973r  Cooke 
and  Bunt  1974],  These  proposals  suggest  studying  the 
cognitive  processes  involved  in  programming  with  a  very 
small  number  of  subjects  using  the  Newell  and  Simon  approach 
to  human  problem  solving  [Newell  and  Simon  1972].  The  first 
paper  [Brooks  1973]  oroposes  the  collection  of  protocol  data 
from  a  single  subject  solving  23  programming  problems  and 
using  it  to  write  a  program  that  models  his  behaviour.  The 
second  paper  [Cooke  and  Bunt  1974]  proposes  eye  movement 
studies,  perception  memory  studies  using  a  tachistosc ope , 
and  protocol  studies  of  programmers  engaged  in  fault 
detection. 

The  preceding  literature  survey  indicates  that  up  until 
now  only  a  handful  of  experimental  studies  have  actually 
been  conducted  with  programmers.  We  decided  to  conduct  our 
research  in  this  area  because  we  thought  worth-while  results 
could  be  obtained.  Our  techniques  and  problems  to  be 
studied,  however,  differ  substantially  from  those  described 


above. 

Unlike 

Filler  and 

S  ime. 

we  have  conducted  our 

research 

with 

experienced 

rather 

than  naive  subjects. 

Unlike 

Goul d , 

Drcngowski , 

and 

Gannon,  we  have  chosen 

primarily 

to 

study  program 

under st andability  and 

modifiability  rather  than  debugging.  Our  work  differs  from 


-19- 


that  proposed  by  Brooks,  Cooke  and  Bunt  in  the  approach 
taken.  Our  studies  reveal  specific  effects  in  large 
populations  of  programmers  without  any  explanation  of  why 
the  effects  occur.  The  studies  of  Brooks,  Cooke  and  Bunt 
may  produce  theories  to  explain  the  behaviour  of  individual 
subjects. 
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Chapter  3 

DESCRIPTION  AND  DISCUSSION  OF  THE  METHODOLOGY 

This  chapter  presents  and  discusses  a  specific 
methodology  for  studying  the  psychological  complexity  of 
computer  programs.  Section  3. 1  contains  an  overview  of  the 
methodology  that  we  suggest  using  for  future  experiments. 
The  methodology  was  arrived  at  as  a  result  of  ten 
experiments  that  we  conducted  using  students  in  various 
computer  science  courses  at  the  University  of  Toronto. 
Detailed  information  about  these  experiments  is  given  in 
Appendices  A  -  J.  Although  we  have  not  conducted  an 
experiment  with  exactly  the  methodology  as  presented, 
several  of  our  experiments  used  close  approximations. 

Cur  last  three  experiments,  eight,  nine,  and  ten 
contained  all  of  our  recommended  components  but  with  a  time 
constraint  of  50  minutes,  as  will  be  discussed  in  section 
3.2.8.  In  our  sixth  experiment,  we  did  not  have  the  problem 
with  time  constraints  but  this  experiment  did  not  contain 
the  form  of  modification  that  we  now  recommend.  In 
discussing  the  validity  of  the  methodology,  we  will 
generally  refer  to  experiments  eight,  nine,  and  ten,  and 
also  to  experiment  six,  when  appropriate. 

Section  3.1  contains  an  overview  of  the  methodology  and 
ends  with  a  concise  outline  of  the  methodology  and  the 
administration  of  an  experiment.  Section  3.2  discusses  the 
individual  components  o^  the  methodology,  and  in  some  cases 
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traces  their  development  throughout 
experiments.  Section  3.3  discusses  the 
whole  and  describes  its  capabilities  and 


our  series 
methodology  as 
limitations. 


of 

a 


3 » 1  Overview  of  the  Methodology 

The  first  step  in  setting  up  an  experiment  consists  of 
deciding  which  factors  to  study  and  at  what  levels  to  study 
them.  For  example,  if  the  factor  to  be  studied  were  choice 
of  variable  names,  one  level  might  be  mnemonic  names  and 
another  level  might  be  meaningless  names.  Chapter  4 
presents  a  list  of  possible  program  complexity  factors  and 
discusses  those  that  we  actually  studied  and  at  which  levels 
we  studied  them.  The  number  of  factors  and  levels  chosen 
for  an  experiment  depends  on  the  number  of  subjects 
available  and  the  size  of  effect  to  be  measured.  This  will 
be  discussed  briefly  in  section  3.3. 
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After  the  programs  have  been  chosen,  versions  of  each 
program  must  be  constructed  containing  all  possible 
combinations  of  levels  of  the  factors.  Sometimes,  it  is  not 
possible  to  vary  the  factors  independently  of  one  another 
and,  thus,  compound  factors  must  be  used.  An  example  of 
this  will  be  discussed  in  Chapter  4  in  the  section  on  the 
control  flow  factor. 


Factors  not  being  explicitly  varied  must  be  held 
constant  throughout  all  versions  of  the  programs.  In 
general,  these  constant  factors  should  be  kept  at  or  near 
their  "best"  possible  level.  The  one  exception  to  this  that 
we  recommend  occurs  in  the  case  of  comments.  Except  in  the 
experiments  in  which  this  factor  is  being  explicitly  varied, 
we  recommend  putting  a  single  global  comment  at  the 
beginning  of  the  program,  stating  the  function  of  the 
program.  This  is  to  ensure  that  the  subjects'  understanding 
of  the  program  comes  from  the  program  itself  and  not  from 
the  comments.  In  constructing  the  various  versions  of  the 
programs,  care  should  be  taken  to  ensure  that  the  program 
either  firs  on  a  single  page  or  is  broken  at  reasonable 
points  over  page  boundaries  (e.g.,  subroutine  boundaries). 


The  next  step  consists  of  making  up  a  fairly  difficult 
quiz  about  each  program  and  choosing  two  or  three  specific 
modifications  for  the  subjects  to  make  to  the  programs. 
Then  an  instruction  booklet  must  be  prepared.  A  sample 
booklet  is  contained  in  Appendix  M. 
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The  experiment  is  now  ready  to  be  administered  to 
subjects.  In  the  following  description,  the  rime  lim 
assume  that  the  subjects  will  be  available  for  two  hou 
While  the  limits  can  be  varied,  the  various  sections  of 
experiment  should  have  fixed  time  limits. 
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data  from  the  self-evaluations  must  then  be  analyzed  using 
statistical  techniques  that  will  be  described  in  section 
The  purpose  of  the  statistical  analysis  is  to  look 
for  differences  in  performance  caused  by  the  factors  being 
varied.  Primarily,  significant  effects  that  are  due  to 
single  factor  variations  are  looked  for,  although  the 
analysis  will  also  indicate  when  several  factors  interact. 
If  our  assumption  that  we  can  understand  psychological 
complexity  by  studying  its  component  factors  is  valid,  then 
we  would  expect  that  relatively  few  interactions  will  occur 
between  factors.  When  significant  interactions  do  occur, 
thev  should  be  investigated  further. 


After  the  results  of  an  experiment  have  been  analyzed, 
further  experiments  with  the  same  factors  must  be  conducted 
with  different  subjects  and  the  results  of  The  experiments 
combined.  This  is  to  ensure  that  results  obtained  are  due 
to  the  factors  being  studied  and  not  peculiarities  of  a 
particular  experimental  situation. 


In  brief,  the  methodology  consists  of  the  following: 

1.  Choose  factors  to  study  and  levels  at  which  to  study 
them. 

2.  Pick  programs  to  use. 

3.  Construct  versions  of  each  program  containing  all 
possible  combinations  of  levels  of  the  factors, 

4.  Construct  a  quiz  about  each  program  and  choose 
modifications  for  the  subjects  to  make  to  the  program. 
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5.  Prepare  the  instruction  booklet. 

6.  Administer  the  experiment. 

7.  Analyze  the  data  obtained  from  the  experiment. 

8.  Conduct  further  experiments  with  the  same  factors  and 
combine  the  results. 

The  experiment  itself  is  administered  as  follows: 

1.  First  Phase 

a)  subjects  read  the  program 

b)  subjects  are  asked  for  a  self-evaluation  of  their 
understanding 

c)  subjects  are  given  a  quiz  about  the  program 

2.  Second  Phase 

a)  subjects  study  the  program  in  detail 

b)  subjects  are  asked  for  a  self-evaluation  of  their 
understanding 

c)  subjects  are  given  the  same  quiz  about  the  program 

3.  Third  Phase 

a)  subjects  are  asked  to  make  specific  modifications 
to  the  program 

b)  subjects  are  asked  for  a  self-evaluation  of  their 
understanding 

c)  subjects  are  given  the  same  quiz  about  the  program 
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3.2  Components  of  the  Methodology 

3.2.1  The  Tx£er im^nt al  Design  and  Statistical  Analysis 

An  experimental  design  is  an  organization  of  an 
experiment  to  which  statistical  tools  can  be  applied  in 
order  to  sort  out  the  effects  of  the  various  factors  being 
studied,  the  interactions  among  the  factors,  and  random 
fluctuations,  and  to  determine  their  significance.  Cur 
experimental  design  and  the  statistical  analysis  technigues 
are  all  well-known.  This  section  contains  a  brief 
de  scription. 

The  basic  experimental  design  is  known  as  a  factorial 
design  [Cochran  and  Cox  1958].  This  consists  of  several 
distinct  factors,  each  at  several  levels.  A  single 
treatment  consists  of  each  of  the  factors  at  one  of  its 
possible  levels.  The  entire  set  of  treatments  consists  of 
all  combinations  that  can  be  formed  from  the  various  levels 
of  the  different  factors.  For  example,  if  we  had  four 
factors,  each  at  two  levels,  then  we  would  have  24  =  16 
different  treatments.  Such  an  experiment  would  be  called  a 
24  factorial  experiment.  If  we  had  two  factors,  one  at  two 
levels  and  one  at  three  levels,  we  would  have  a  (2x3) 
factorial  experiment  with  6  different  treatments. 

A  single  replicate  of  an  experiment  consists  of  having 
each  possible  treatment  assigned  to  a  different  subject. 
Thus,  a  single  replicate  of  a  24  factorial  experiment  would 


-27- 


consist  of  16 

subjects,  each 

rece iving 

a  different 

treatment.  In 

order  to  obtain 

meaningful 

results,  we 

generally  need 

several  replicates 

of  an  experim 

ent.  In  this 

case,  subjects 

receiving  the  same 

treatment  ar 

e  referred  to 

as  being  in  the  same  cell. 

By  using  a  factorial  design,  we  can  measure  the  main- 
effects  of  the  factors  as  well  as  the  interactions  among 
them.  Two  factors  A  and  B  are  said  to  have  no  interaction 
if  the  difference  between  the  observations  corresponding  to 
any  two  levels  of  A  is  the  same  for  all  levels  of  3.  If 
this  is  not  the  case,  then  A  and  E  are  said  to  interact.  As 
mentioned  in  section  3.1,  we  would  hope  that  relatively  few 
interactions  occur,  but  using  a  factorial  design  will  permit 
us  to  determine  the  interactions  that  actually  do  occur. 
Other  experimental  designs  (e.g.,  latin  squares  [Cochran  and 
Cox  1  958  ])  permit  the  use  of  fewer  subjects,  but  such- 
designs  do  not  permit  the  analysis  of  all  possible 
interactions  among  factors. 


A  statistical  technique  known  as  analysis  of  variance 
[ Hoel  1962]  is  used  to  analyze  the  results  of  a  factorial 
experiment.  This  technique  consists  of  determining  the 
components  of  the  total  variance  of  the  experiment.  These 
consist  of  the  variances  associated  with  all  the  main 
effects,  the  variances  associated  with  all  the  interactions, 
and  the  variance  associated  with  random  fluctuations  within 
cells.  An  effect  is  stat istically  significant  (unlikely  to 
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have  occurred  by  chance)  only  if  its  associated  variance  is 
sufficiently  larger  than  the  within  cells  variance. 

The  first  step  in  this  procedure  consists  of  computing 
the  grand  mean  of  all  the  observations,  x.  For  the  purpose 
of  the  rest  of  this  discussion,  we  will  deal  wita  an 
experiment  having  two  factors  with  I  and  J  levels, 
respectively,  and  F  replicates.  Observation  x-jjr  would 
refer  to  a  treatment  at  the  ith  level  of  the  first  factor, 
the  jth  level  of  the  second  factor,  and  in  the  rth 
replicate.  The  analysis  can  easily  be  extended  to  more 
factors  as  will  be  explained  later. 


The  next  step  of  the  procedure  consists  of  representing 
the  total  sum  of  squares  of  differences  of  the  observations 
x^jr  from  their  mean  x  (i.e. ,  the  total  variance)  in  the 
following  form: 


R  J 

l  l 

r=  1  j  =  l 


I 


l 

i  =  l 


wheiS  x.  is  the  mear  of  all 

l  •  • 

first  factor. 


R  J  I 

I  I  I  [Oi. 


-  x)2 


II 

i—1 

< _ i. 

II 

1  i  =  l 

-L 

+ 

(x  . 

•  J  • 

-  x)2 

+ 

(x.  . 

1J  • 

-  x .  -  X  .  +  x) 2 

i  .  .  .  j  .  J 

+ 

•1 - > 

•H 

w 

-  X. -  )2] 

1J  • J  J 

ob  ser vat  ion  s 

at  level  i  of  the 

X  . 

*  3  ' 
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j  of  the  second  factor,  and  x  —  .  is  the  cell  mean  for 
treatment  at  level  i  of  the  first  factor  and  level  j  of 
second  factor. 

The  first  term  in  this  sum  represents  the  sum 

squares  associated  with  the  main  effect  of  the  first  fact 
The  second  term  in  the  sum  represents  the  sum  of  squa 
associated  with  the  main  effect  of  the  second  factor, 
third  term  represents  the  sum  of  squares  associated  with 
interaction  effect  of  the  two  factors.  The  final  t 

represents  the  sum  of  squares  within  cells. 

The  variance,  or  mean  square,  for  each  of  these  effe 
is  then  computed  by  dividing  the  sum  of  squares  by 
number  of  degrees  of  freedom  for  the  effect.  The  degrees 
freedom  associated  with  any  component  of  an  experiment 
the  number  of  independent  parameters  required  to  descr 
that  component  in  the  model.  For  a  main  effect,  the  num 
of  degrees  of  freedom  is  equal  to  the  number  of  levels 
the  factor  minus  one.  For  an  interaction  effect,  the  num 

of  degrees  of  freedom  is  equal  no  the  product  of  the  num 

of  degrees  of  freedom  in  each  of  the  factors  involved  in 
interaction.  For  the  within  cells  variance,  the  number 
degrees  of  freedom  is  equal  to  the  product  of  the  number 
cells  and  the  number  of  subjects  per  cell  minus  one. 

A  quantity  known  as  the  F-ratio  is  then  computed 
each  effect.  The  F-ratio  is  the  ratio  of  the  varia 
associated  with  the  effect  to  the  variance  within  cel 


the 

the 

of 

or. 

res 

The 

the 

erm 

cts 

the 

of 

are 

ibe 

ber 

of 

ber 

ber 

the 

of 

of 

f  or 
nee 
Is. 


-30- 


Tables  of  F- distr ibut ions  [Abramowitz  and  Stegun  1964]  are 
rhen  used  to  determine  the  significance  level  of  the  effect 
from  the  F-ratio.  An  effect  that  is  significant  at  <5% 
level  is  one  that  is  likely  to  occur  randomly  less  than  once 
in  twenty  times.  An  effect  that  is  significant  at  <1%  level 
is  one  that  is  likely  to  occur  randomly  less  than  once  in  a 
hundred  times.  The  larger  the  F-ratio,  the  more  significant 
the  effect. 


^his  analysis  is  easily  extended  to  consider  more  than 
two  factors;  however,  higher-order  interactions  must  be 
considered  in  addition  to  the  main  effects  and  two-way 
interactions.  For  example,  if  we  had  three  factors,  the 
total  sum  of  squares  would  have  eight  terms  in  it:  one  term 
for  each  of  the  three  main  effects;  one  term  for  each  of  the 
three  two-way  interactions;  one  term  for  the  within  sells 
variance;  and  the  following  term  for  the  three-way 
interaction : 
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When  dealing  with  a  factor  with  more  than  two  levels, 
the  analysis  of  variance  as  described  above  will  only 
indicate  when  a  significant  difference  has  occurred,  but 
will  not  indicate  between  which  levels  the  difference  is 
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significant.  In  order  to  determine  this  in  our  experiments, 
analysis  of  variance  can  be  performed  on  subsections  of  the 
data,  considering  only  two  levels  of  each  factor  at  a  time. 
This  is  permissible  because  any  single  subject  is  only  given 
one  treatment  and  is  not  affected  by  the  number  of  levels  of 
the  various  factors  that  exist  in  other  treatments.  Of 
course,  this  procedure  of  making  several  comparisons  affects 
the  significance  level.  A  nominal  5^  significance  level 


corresponds 

to  a  slightly 

high 

er 

level  b 

eca  use 

of  the 

multiplicity 

of  comparisons. 

Another 

t echnigu e  can  be 

used 

to 

c  ombine 

the  re 

suits  of 

similar  expe 

riments.  This  is 

done 

because  an 

effect 

showing 

up  mildly  significant  in  several  experiments  may  actually  be 
more  significant  than  any  experiment  separately  would 
indie  ate . 

The  technique  consists  of  computing  the  significance 
level  for  each  effect  from  its  E-ratio  for  each  of  the 
experiments  to  be  combined.  Each  significance  level  is  then 
converted  to  a  chi-square  value,  choosing  a  number  of 
degrees  of  freedom  proportional  to  the  relative  importance 
of  the  various  experiments.  The  chi-square  values  from  the 
different  experiments  are  then  totaled,  for  each  effect. 
The  chi-square  sums  are  then  converted  back  to  significance 
levels,  vrith  the  number  of  degrees  of  freedom  being  the 
total  of  the  number  of  degrees  of  freedom  assigned  to  each 
of  the  individual  effects.  (This  transformation  of 
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coordinates  is  useful  because  the 
distributions  is  again  a  chi-sguare 
the  sum  of  F  distributions  is  not  an  F 


sum  of  chi-squ 
distribution,  wher 
distribution. ) 


In  evaluating  the  methodology,  we  want  to  do  more  t 
evaluate  the  significance  of  individual  results.  We  a 
want  to  ensure  -that  we  are  getting  more  results  than  wo 
be  expected  by  chance.  A  statistical  technique  can  be  u 
to  determine  if  something  happens  significantly  m 
frequently  than  would  be  expected  by  chance  (e.g., 
number  of  significant  results  obtained  on  any  measur 
This  technique  consists  of  comparing  the  actual  number 
occurrences  with  the  number  expected  to  occur  purely 
chance  and  then  using  the  binomial  distribution  to  determ 
the  significance  level  [  Hoel  1962  ]. 


Finally,  statistical  techniques  can  be  used  to  red 
within  cells  variance.  These  will  be  discussed  in  sect 
3.3.  We  used  one  of  these  techniques,  but  only  on 
second  experiment.  Before  performing  the  analysis 
variance,  a  least  sguares  fit  was  performed  with  a 
subject's  final  grade  in  the  computer  science  course  f 
which  the  subjects  were  obtained,  and  his  difference  f 
the  cell  mean  in  the  experiment.  This  effect  was  t 

subtracted  from  his  performance  in  an  effort  to  reduce 
within  cells  variance  due  to  the  differing  abilities  of 
subjects  as  measured  by  their  grades.  This  technique 
abandoned  after  the  second  experiment,  however,  par 
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because  it  only  slightly  improved  the  results  obtained  in 
the  experiment,  and  partly  because  several  subjects  in  the 
later  experiments  insisted  on  remaining  anonymous. 

3.2.2  Instructions  for  Study  of  Programs 

One  of  our  fundamental  assumptions  is  that  each  program 
has  an  intrinsic  complexity  but  that  a  programmer's  skill 


and 

study 

of 

th  e 

program  also  contribute  to 

his 

under  standing 

of 

it.  One 

component  of  our  methodology 

that 

has 

evolved 

thr 

ough  out 

our  series  of  experiments  is 

the 

method  by  which  subjects  are  told  to  study  the  programs. 

In  the  first  two  experiments  that  we  conducted,  the 
subjeers  were  instructed  to  study  the  program  by  reading  and 
hand- simulating  it  for  thirty  minutes  and  keeping  track  of 
the  values  of  all  variables.  We  then  attempted  to  measure 
their  understanding.  We  decided  to  refine  this  procedure  in 
the  third  experiment  by  having  the  subjects  read  the  program 
in  the  first  phase  and  then  having  them  hand-simulate  in  the 
second  phase.  Subjects  were  allowed  to  work  at  their  own 
pace,  having  been  told  the  total  time  allotted  for  the 
experiment.  This  did  not  work  out  well  and  will  be 
discussed  in  section  3.2.8.  In  the  fourth  experiment,  we 
used  the  same  instructions  as  in  the  third,  except  that  we 
used  fixed  time  limits  for  the  various  sections.  We  also 
added  a  new  phase  in  this  experiment,  having  the  subjects 
modify  the  program.  We  then  attempted  to  measure  their 
understanding  after  each  of  the  three  phases. 


-34- 


The  major  problem  with  hand- simulation  was  that  the 
subjects  complained  about  being  forced  to  do  it  and  felt 
that  they  should  be  allowed  to  study  the  program  any  way 
they  wished.  Consequently,  in  our  fifth  experiment,  we 
decided  to  investigate  the  method  by  which  the  subjects 
studied  the  program.  The  first  phase  of  the  experiment 
consisted  of  having  the  subjects  read  the  program,  as 
before,  but  in  the  second  phase,  we  varied  the  method  by 
which  the  subjects  were  instructed  to  study  the  program. 
The  first  group  of  subjects  was  instructed  to  hand-simulate 
rhe  program  as  in  the  previous  experiments.  The  second 


group  was 

instructed 

to  read 

a  machine-generated 

source- 

language 

trace 

[  Satt 

erth wait 

e  1972  ]  of  the 

program 

in  which 

each  instruction 

was 

traced 

the  first  two 

times 

it  was 

execu  ted . 

The 

sub  j 

ects  in 

the  third  group 

were  in 

struc ted 

to  study 

the  program 

any  way 

they  wished  and 

to  describe  the 

method  they  used. 

The  main  outcome  from  this  experiment  was  that  no 
significant  difference  appeared  among  the  three  methods  of 
study  on  either  the  self-evaluations  or  the  quizzes.  On  the 
self-evaluation  given  immediately  after  the  detailed  study, 
the  mean  scores  for  the  three  levels  of  study  were:  6.25, 

6.5,  6.C.  On  the  corresponding  guiz,  the  mean  scores  were: 

7.5,  7.6,  6.9.  This  is  not  conclusive,  since  we  were 
working  with  a  small  sample,  but  it  at  least  indicates  that 
the  differences  were  not  large.  Most  of  the  subjects  who 
were  allowed  to  study  the  program  as  they  wished  used  some 
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sort  of  hand-simulation, 
were  given  the  trace  found 
hand- simulation ,  although 
oarticular  trace  used,  not 


In  fact,  some 

of  the 

sub  jec ts 

who 

it  inadequate 

and 

resorted 

to 

this  may  be 

the 

fault  of 

rhe 

of  the  method  in  general. 


As  a  result  of  this  experiment  and  the  complaints  of 
the  subjects  about  being  forced  to  hand- simulate,  we  decided 
that  it  would  be  reasonable  in  the  second  phase  of  an 
experiment  to  allow  the  subjects  to  study  the  program  in 
detail  using  any  method  they  wished.  In  our  sixth 
experiment,  we  did  reguire  the  subjects  to  hand-simulate, 
but  only  because  we  administered  this  experiment  before  we 
had  analyzed  the  results  of  the  fifth  experiment. 

3.2.3  The  Subjective  Measure  of  Understanding 

One  of  our  assumptions  concerning  complexity  is  that  it 
affects  a  person’s  ability  to  understand  a  program.  One  way 
to  measure  at  least  a  component  of  a  subject’s  understanding 
of  a  program  is  to  ask  him  how  well  he  understands  it.  The 
particular  subjective  measure  of  understanding  that  we  use 
is  the  subject's  self-evaluation  of  his  understanding  of  the 
program.  The  subject  is  asked  to  circle  a  number  from  3  to 
9  indicating  how  well  he  feels  he  understands  the  program, 
where  0  means  "not  at  all",  3  means  "I  can  explain  what  it 
does”,  6  means  "I  can  explain  why  it  works",  9  means  "I 
understand  it  perfectly",  and  the  other  numbers  indicate 
intermediate  states  between  these . 
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This  is  the  one  component  of  our  methodology  that 
remained  unchanged  throughout  our  series  of  experiments. 
The  only  change  that  we  made  is  that  in  the  early 
experiments  we  only  obtained  one  self-evaluation,  and  in  the 
later  experiments  we  obtained  a  self-evaluation  after  each 
of  the  three  phases. 

This  measure  is  very  likely  to  be  biased  by  the 
subjects'  backgrounds  and  attitudes.  Nonetheless,  it  is  an 
important  measure  because  a  programmer's  opinion  of  his 
understanding  of  a  program  will  undoubtedly  affect  his  work 
effort  on  it  regardless  of  why  he  holds  that  opinion.  As 
will  be  discussed  later,  however,  we  do  recommend  conducting 
experiments  in  which  the  background  of  the  subjects  is 
•created  as  an  additional  factor  so  that  its  effect  on  our 
measures,  in  particular  self-evaluation,  can  be  assessed. 

We  feel  confident  about  the  self-evaluation  measure  for 
several  reasons.  We  obtained  results  with  it  that  generally 
confirmed  our  intuitive  impressions.  Also,  the  number  of 
significant  results  obtained  with  this  measure  was 
significantly  more  than  would  be  expected  by  chance. 
Considering  only  main  effects  on  experiments  six,  eigat, 
nine,  and  ten,  which  closely  followed  our  proposed 
methodology,  there  were  21  possible  results  that  could  have 
occurred.  Of  these  21,  10  were  significant  at  <5%  level. 
We  would  expect  1.05  to  occur  purely  by  chance.  Considering 
Ahese  results  as  independent  and  using  the  binomial 
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distribution  ,  1C  is  significantly  greater  than  1.05  out  of 
21  at  <.1%  level.  In  addition,  as  will  be  seen  in  Chapter 
4,  the  results  obtained  in  experiment  eight  were 
consistently  repeated  in  experiment  ten,  which  was  virtually 
the  same  experiment  administered  to  a  different  group  of 
subjects.  In  fact,  all  the  effects  on  variable  names  on 
this  measure  in  experiment  eight  agreed  in  direction  with 
the  effects  on  experiment  ten  and  many  of  them  were 
statistically  significant  on  both  experiments.  (There  were 
no  other  significant  effects  in  either  experiment  on  this 
measure  except  an  unexplained  three-way  interaction  on  the 
first  self-evaluation  of  experiment  eight.) 

3.2.4  The  Object ive  Measure  of  Understanding 


In  addition  to  subjectively  measuring  a  subject's 
understanding  of  a  program,  it  is  also  desirable  to  measure 
it  objectively.  Obtaining  an  objective  measure  of  an 
abstract  concept  such  as  understanding  is  not  easy  and 
consequently  this  component  of  our  methodology  underwent 
considerable  change  throughout  our  series  of  experiments. 


In  our  first  two  experiments,  we  used  two  objective 
measures  of  understanding,  how  well  the  subjects  hand- 
simulated  the  program  and  how  well  they  were  able  to  fill  in 
blanks  in  a  paragraph  describing  the  program.  In  our  third 
experiment,  we  abandoned  the  hand-simulation  measure. 
Although  we  feel  that  hand-simulation  is  an  important 
process  that  can  contribute  to  one's  understanding  of  a 
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program,  we  do  not  consider  it  a  valid  measure  of 
understanding.  This  is  because  we  found  that  in  our  first 
two  exDeriments  subjects  said  they  had  put  themselves  in 
'•machine-mode"  and  hand- simulated  perfectly  without  showing 
any  understanding  of  the  program  on  our  other  measures. 

In  the  third  experiment,  we  also  refined  the  fill-in- 
the-blanks  measure  by  replacing  it  with  a  ten  question  quiz 
about  the  program.  We  did  this  because  questions  were 
easier  no  prepare  than  a  paragraph  with  missing  words.  In 
addition,  our  quizzes  were  constructed  so  as  to  induce  a 
detailed  study  of  the  program.  For  example,  question  five 
might  follow  from  questions  two,  three,  and  four  and  thus 
direct  the  attention  of  the  subjects  in  studying  the 
program.  Although  we  recommend  a  ten  question  quiz  for  the 
size  of  program  that  we  used,  this  can  be  altered  slightly. 
Using  too  few  questions  (e. g. ,  one  or  two),  however,  would 
not  provide  enough  discrimination  between  subjects  and  using 
a  large  number  (e.g.,  100)  would  consume  additional  time 
without  providing  much  additional  discrimination. 


The  ability  to  answer  reasonable  questions  about  a 
program  is  certainly  a  dimension  of  understanding.  In 
experiments  three  through  seven,  however,  we  did  not  acaieve 
any  statistically  significant  results  on  the  quizzes.  This 
was  due,  we  felt,  to  the  fact  that  they  were  too  easy  and 
nearly  everyone  obtained  high  scores.  In  experiments  eight 
through  ten,  we  made  the  quizzes  much  more  difficult. 
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Although  one  significant  result  did  appear,  the  quiz  scores 
in  general  were  very  low.  This  may  have  been  due  to  the 
^act  that  these  three  experiments  were  limited  to  50 
minutes.  If  the  subjects  had  had  more  time  to  study  the 
programs,  other  significant  differences  may  well  have  shown 
up  on  the  quizzes.  Only  further  experiments  will  confirm 
this. 

We  do  have  evidence,  however,  that  the  quizzes  are  in 
fact  measuring  understanding,  despite  the  fact  that  very  few 
significant  results  appeared.  On  experiments  eight,  nine, 
and  ten,  there  were  24  possible  pair-wise  comparisons  that 
could  be  made  between  levels  of  factors,  considering  only 
main  effects.  Or.  18  of  these  24  comparisons,  the  difference 
between  the  mean  scores  was  in  the  same  direction  as  the 
difference  on  the  corresponding  self-evaluation  measure. 
Purely  by  chance,  we  would  expect  only  half  (i . e . ,  12)  of 
the  effects  to  agree  in  direction.  Considering  these 
effects  as  independent  and  using  the  binomial  distribution, 
18  is  significantly  greater  than  12  out  of  24  at  <1%  level. 
This  indicates  that  the  same  effects  that  are  showing  up  on 
the  self-evaluations  are  also  showing  up  on  the  quizzes  but 
much  smaller  in  magnitude. 


This 

point  is  also 

supported  by 

our 

analysis 

of 

programmer 

variability,  and 

our 

an  al ysis 

of 

the  size 

of 

effect  that  would  have  shown 

up 

significantly. 

This  will 

be 

presented 

in  detail  in  section 

3.3.  The 

re 

suit  of  t 

his 

-40- 


analysis  indicates  that  with  the  number  of  subjects  that  we 
used  in  our  last  three  experiments,  and  the  small  within 
cells  variance  that  we  consistently  obtained  on  the  quizzes, 
an  absolute  difference  of  two  units  between  any  two  levels 
of  any  factors  would  have  been  significant.  This  supports 
our  suspicion  that  the  constrained  time  limits  (and  thus  the 
very  low  mean  scores)  accounted  for  some  of  the  scarceness 
of  significant  differences  on  the  quizzes.  We  cannot, 
however,  rule  out  the  possibility  that  the  particular 
factors  and  levels  chosen  would  not  make  a  significant 
difference  (even  of  merely  two  units)  on  the  quizzes  under 
any  circumstances. 


There  is  one  additional  piece  of  evidence  to  support 
quizzes  as  an  objective  measure  of  understanding. 
Experiment  ten  was  essentially  a  repeat  of  experiment  eight 
with  a  less  experienced  group  of  subjects.  We  combined  the 
data  from  these  experiments  and  performed  an  analysis  of 
variance,  using  the  course  in  which  the  subjects  were 
enrolled  as  an  additional  factor.  On  the  second  (and  final) 
quiz,  the  subjects  in  the  second-year  class  performed 
significantly  better  than  the  subjects  in  the  first-year 
class  (<1^  level).  Programming  experience  is  undoubtedly 
one  means  of  mastering  complexity.  In  particular,  the  more 
experience  a  subject  has,  the  easier  it  should  be  for  him  to 
understand  a  new  program  when  presented  to  him.  Thus,  the 
fact  that  our  quizzes  significantly  differentiated  between 


subjects 
they  may 


with  different  levels  of  experience  indicates  that 
be  measuring  understanding. 


One  point  about  the  quizzes  needs  explanation:  the  use 
of  the  same  quiz  after  all  three  phases  of  the  experiment. 
There  are  two  reasons  for  doing  this.  The  first  reason  is 
that  it  would  be  fairly  difficult  to  make  up  three 
independent  quizzes  for  the  same  program.  The  other  reason 
is  that  our  approach  enables  us  to  perform  the  analysis  of 
variance  to  determine  the  effects  of  the  factors  being 
studied  at  the  end  of  each  of  the  three  phases  of  the 
experiment  and  also  to  get  a  crude  measure  of  overall 
improvement  throughout  the  experiment.  This  crude  measure 
can  be  obtained  by  comparing  the  grand  means  of  the  three 
quizzes,  but  there  is  no  way  to  determine  whether  the 
difference  in  score  is  due  to  the  additional  study  or  to  the 
fact  that  the  same  quiz  is  being  repeated.  An  alternative 
approach  is  to  systematically  give  each  subject  a  different 
sub-section  of  the  quiz  at  each  of  the  three  points  in  the 
experiment,  with  random  assignments  made  to  subjects.  This 
would  enable  us  to  ascertain  the  improvement  in  performance 
due  to  the  additional  study,  but  would  greatly  complicate 
the  analysis  of  variance  of  the  effects  of  the  factors  being 
studied,  which  is  the  primary  goal  of  the  experiment. 

3.2.5  Modifications  to  the  Program 


As  mentioned  in  the  introduction,  one  of  our 
assumptions  is  that  the  modifiability  of  a  program  is 
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affected  by  its  complexity.  Consequently,  it  is  import 
to  measure  the  modifiability  of  a  program  as  well  as 
under st an debility. 

Thus,  in  *he  fourth  experiment  that  we  conducted, 
added  the  third  phase  to  the  methodology,  having 
subjects  modify  the  program.  In  this  experiment,  and 
next  two,  we  had  a  great  deal  of  difficulty  in  evaluat 
the  modifications  made  by  subjects  because 
specif ications  given  were  very  general.  We  finally  deci 
to  mark  them  on  a  five-point  scale:  fully  correct,  aim 
correct,  right  idea,  not  very  good,  and  totally  wrong.  I 
did  not  prove  satisfactory  because  the  general  nature  of 
modifications  enabled  the  subjects  to  provide  modificati 
that  were  not  easily  quantifiable  even  with  this  five-po 
scale.  Often,  several  alternative  approaches  to 
modification  could  be  taken  and  thus  it  was  difficult 
compare  and  mark  various  solutions.  Consequently, 
decided  to  alter  this  component  of  the  methodology. 


On  the  eighth,  ninth,  and  t 
the  subjects  to  make  two  or  three  s 
the  program.  We  used  three  type 
first  was  of  the  form:  "If  lines  x 
to  the  following,  what  other  chan 
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would  have  to  be  changed  in  order  t 
f  ollowing?" 
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The  third  was  of  the  form: 


"make  modifications 
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within  lines  xx  through  yy  in  order  to  make  the  program  do 
the  following.1’ 


This  procedure 

worked  very  w 

ell. 

The 

specific  nat 

ure 

of  the  modifications 

required  made 

them 

very 

easy  for  us 

to 

evaluate.  It  is 

not  clear 

whether 

these  speci 

f  ic 

modifications  are 

more  typical 

of 

actual  programm 

ing 

situations  than  our  earlier  type  of  general  modification. 
They  are,  however,  much  more  easily  quantified  for 
controlled  experimental  studies. 

We  obtained  some  statistically  significant  results  on 
the  modifications,  which  will  be  discussed  in  Chapter  4.  In 
addition,  we  compared  the  direction  of  the  effects  on  the 
modifications  with  the  direction  of  the  effects  on  the 
quizzes  given  immediately  after  the  modifications.  Out  of 
28  effects,  21  were  in  the  same  direction.  Purely  by 
chance,  we  would  expect  only  1 4  of  the  effects  to  agree  in 
direction.  Considering  these  effects  as  independent  and 
using  the  binomial  distribution,  21  is  significantly  greater 
than  14  at  <.5%  level.  This  evidence  supports  our 
assumption  that  complexity  affects  under standabil if y  and 
modifiability  consistently  as  measured  by  our  quiz  and 
modification  scores. 

The  one  change  we  recommend  to  this  procedure  in  future 
experiments  is  to  enforce  separate  time  limits  for  each  of 
the  modifications  in  order  to  ensure  a  uniform  partitioning 
of  time  amoncr  modifications  by  the  various  subjects.  Using 
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current  procedure,  it  is  more  reasonable  to  sum  the 
s  from  the  individual  modifications  before  performing 
analysis  of  variance.  We  did  in  fact  do  this,  but  have 
n  to  present  +he  results  of  the  analysis  of  the 
idual  modifications  because  no  effects  appeared  on  the 
d  scores  that  had  not  appeared  on  individual  scores, 
in  fact  some  effects  vanished  when  the  scores  were 
d. 

3.2.6  finding  Errors  in  the  Program 


In  our  seventh  experiment  we  took  a  digression  from  our 
type  of  experiment.  In  this  experiment  we  studied  a 
ct's  ability  to  find  errors  in  a  program.  We  seeded 
program  with  two  logic  errors.  The  factor  that  we 
d  in  this  experiment  was  the  subject's  method  of  study, 
ubjects  were  given  fifteen  minutes  to  read  the  program, 
they  were  told  might  contain  errors  and 
iciencies,  and  ten  minutes  for  a  self-evaluation  and 
They  were  next  given  twenty-five  minutes  for  a 
led  study  of  the  program.  One  group  was  told  to  study 
program  using  any  method  they  wished.  A  second  group 
iven  a  machine-generated  source  language  trace,  as  in 
fifth  experiment.  The  third  group  was  given  the  trace 
11  as  a  profile  of  the  program  showing  the  number  of 
each  statement  had  been  executed.  After  the  detailed 
,  the  subjects  were  given  ten  minutes  for  a  second 
evaluation  and  quiz  and  then  forty-five  minutes  to 
ct  the  errors  and  inefficiencies  in  the  program. 
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Nothing  statistically  significant  showed  up  on  the 
quizzes,  self-evaluations,  or  number  of  errors  found.  This 
is  not  too  surprising  in  light  of  our  fifth  experiment  and 
the  studies  at  Yorktown  Heights  [Gould  and  Drongowski  1972, 
Gould  1973  ]. 

We  had  a  problem  scoring  quizzes,  since  some  subjects 
answered  the  questions  about  the  program  with  the  errors  it 
contained  and  other  subjects  answered  the  questions  about 
their  "corrected"  version  of  the  program.  This  problem 
could  be  eliminated  in  future  experiments  by  making  the 
directions  more  explicit. 

This  was  the  only  experiment  in  which  we  attempted  to 
study  a  subject's  ability  to  find  errors  in  a  program.  As 
we  mentioned  in  the  introduction,  we  concentrated  primarily 
on  developing  a  methodology  for  measuring  understandability 
and  modifiability. 

3.2.7  The  Questionnaire 

At  the  end  of  each  experiment  we  gave  the  subjects  a 
questionnaire.  A  sample  questionnaire  is  contained  in  the 
sample  instruction  booklet  in  Appendix  M. 

'’’he  main  purpose  was  to  ask  the  subjects  about  any 
specific  difficulties  they  had,  whether  they  had  seen  the 
program  before,  and  any  general  comments  they  wished  to 
make.  We  did  this  to  ensure  that  a  subject's  score  was  not 


due  to  prior  familiarity  with  the  program  or  difficulty  with 
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specific  language  constructs.  No  responses  from  the 
questionnaires  indicated  that  our  experiments  were  being 
affecxed  by  such  problems. 

In  addition,  we  also  asked  the  subjects  to  rank  the 
helpfulness  of  various  factors  on  a  scale  of  0  to  9.  In  our 
early  experiments,  we  were  careful  to  include  all  the 
factors  being  varied  as  well  as  several  irrelevant  factors. 

In  the  second  experiment,  we  analyzed  the  lata 
concerning  the  helpfulness  of  the  various  factors  to  see 
whether  the  subjects  perceived  the  differences  we  thought  we 
were  inse rting.  In  this  experiment  we  studied  three  factors 
and  in  all  three  cases,  the  predicted  effect  was  significant 
at  < . 1 ?  level.  The  subjects’  ranking  indicated  that 
comments  were  significantly  found  to  be  helpful  in  the 
programs  containing  extensive  comments,  with  mean  scores: 
5.38  (for  programs  containing  extensive  comments)  vs.  0.8 
(for  programs  not  containing  extensive  comments) . 
Similarly,  paragraphing  was  ranked  as  being  more  helpful  in 
the  programs  containing  paragraphing:  5.29  vs.  1.04,  and 
variable  names  were  ranked  as  being  as  being  more  helpful  in 
the  programs  containing  mnemonic  variables:  6.46  vs.  1.25. 
These  resul+s  were  deemed  a  sufficient  validation  of  this 
aspect  of  the  methodology,  and  this  type  of  analysis  was 
not,  in  general,  repeated  in  other  experiments. 

We  did  not,  however,  delete  this  section  of  the 
questionnaire  and  thus  continued  to  collect  the  data, 
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ugh  we  had  decided  not  to  analyze  it.  Just  to  m 
in  that  this  was  a  valid  decision,  we  did  analyze 
concerning  the  helpfulness  of  paragraphing 
iment  nine.  Again,  the  predicted  effect 
ficant  at  <.1%  level,  paragraphing  was  ranked  as  be 
helpful  in  the  programs  containing  paragraphing:  4 

.63  . 


3.2.8  Administrative  Details 


Certain  administrative  details  of  the  methodology  n 
discussed.  Some  of  them  may  seem  trivial,  but  of 
ngly  trivial  details  are  symptoms  of  serious  proble 
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effects  due  to  replicate  out  of  a  possible  21  effects,  but 
this  is  not  significantly  more  than  would  have  been  expected 
by  chance,  even  at  <10%  level. 

We  always  had  an  extra  replicate  of  experimental 
materials  prepared  so  that  extra  subjects  who  showed  up 
would  be  kept  busy.  In  addition,  such  subjects  could  be 
substituted  for  other  subjects  if  the  need  arose  to 
disqualify  a  subject  (e. g. ,  a  subject  had  previously  seen  a 
particular  program) . 

Another  important  aspect  of  our  methodology  is  the 
enforcement  of  fixed  time  limits  for  the  various  phases  of 
♦  he  experiment.  In  experiment  three,  subjects  were  allowed 
to  work  at  their  own  pace,  having  been  told  the  total  time 
allotted  for  the  experiment,  and  told  to  emphasize  accuracy 


over  speed. 
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a  result  did  not  finish  the  experiment.  In  addition, 
several  subjects  failed  to  record  the  time  at  various 
points.  As  a  result,  we  decided  to  abandon  the  approaci  of 
letting  subjects  work  at  their  own  pace  and  instead  enforced 
strict  time  limits  telling  the  subjects  in  advance  how  long 
they  would  have  for  each  section.  We  continued  to  use  the 
approach  of  telling  subjects  to  emphasize  accuracy  over 
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speed  in  order  to  reduce 
different  understandings 
strict  time  limits  enabled 


variations  caused  by  subjects' 
of  what  was  expected.  Enforcing 
us  to  obtain  complete  data. 


Another  thing  that  we  learned  from  experiment  three 
concerned  the  physical  set-up  of  the  experimental  materials. 
Throughout  the  experiment  there  was  an  inordinate  amount  of 
page  flipping,  indicated  by  a  high  noise  level,  and  comments 
by  the  subjects  after  the  experiment  confirmed  this 
impression.  The  problem  was  caused  by  the  fact  that  the 
pages  provided  for  the  subjects  to  keep  track  of  their  hand- 
simulation  were  stapled  together  and  had  to  be  constantly 
flipped.  In  addition,  one  of  the  programs  did  not  fit  on 
one  page  and  was  not  split  at  a  reasonable  point. 


To  avoid  repetitions  of  this  problem,  all  programs  used 
in  later  experiments  either  fit  on  one  page  or  were  broken 
at  reasonable  points,  e.g.,  subroutine  boundaries.  In 
addition,  in  those  experiments  in  which  we  required  hand- 
simulation  we  provided  the  subjects  with  fold-out  sheets 
(i.e.,  the  form  fit  on  one  large  page)  to  keep  track  of  the 
value  of  variables  and  we  also  used  rooms  containing  tables 
for  the  subjects  to  work  at  rather  than  chairs  with  arms. 


One  thing  that  went  wrong  with  experiment  seven  (the 
one  in  which  we  seeded  errors  in  the  programs)  was  caused  by 
a  frustrated  subject.  Part  way  through  the  detailed  study, 
he  called  us  over  to  ask  a  question  and  although  he  thought 
he  was  whispering,  asked  loudly  enough  for  all  the  other 
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subjects  to  hear,  "Shouldn't  this  be  N!  instead  of  N?"  This 
was,  in  fact,  one  of  the  errors  that  we  had  seeded  in  the 
program.  Although  this  partly  ruined  the  experiment,  our 
only  suggestion  for  correcting  it  would  be  to  have  isolation 
booths  or  conduct  the  experiment  separately  with  each 
subject,  neither  of  which  may  be  feasible. 


A  problem  we  experienced  in  experiments  eight,  nine, 
and  ten  was  caused  by  the  fact  that  our  subjects  were  only 
available  for  50  minutes.  We  had  developed  the  methodology 
assuming  that  the  subjects  would  be  available  for  about  two 
hours,  but  the  practical  constraint  forced  us  to  modify  our 
procedures.  Pasically,  we  did  this  by  eliminating  the  first 
guiz  and  shortening  the  time  limits  on  the  various  sections 
of  the  experiments.  We  were  not  happy  about  doing  this,  but 
had  no  choice.  We  did  achieve  statistically  significant 
results,  but  very  few  of  them  were  on  the  quizzes  and  this, 
we  feel,  may  have  been  due  to  the  fact  that  the  subjects  did 
not  have  enough  time  to  study  the  programs. 


Several  administrative  problems  arose  in  all  of  the 
experiments  conducted.  One  problem  encountered  was  with  the 
choice  of  programs.  It  turned  out  to  be  rather  difficult  to 
find  programs  that  were  short  enough  to  be  manageable  in  the 
time  allotted  for  the  experiment,  yet  not  trivial  to 
understand.  A  great  deal  of  time  was  spent  searching  for 
such  programs  and  putting  them  in  usable  form.  The 
0.2ilscted  Algorithms  from  CACM  and  various  computer  science 
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text  books  and  journals  provided  many  of  our  programs.  As 
will  be  discussed  in  section  4.2,  choice  of  program  almost 
never  interacted  with  the  other  factors. 

Finally,  a  problem  was  encountered  with  the  choice  of 
subjects.  Ideally,  a  wide  range  of  subjects  with  varying 
backgrounds  should  be  used,  with  the  subjects  arranged  into 
homogeneous  groups  so  that  the  background  of  the  subjects 
could  be  treated  as  an  additional  factor  and  its  effect 
measured.  For  practical  reasons,  we  were  limited  to 
students  in  various  computer  science  courses.  The  only  real 
problem  this  caused  was  on  the  first  experiment  in  which  the 
subjects  had  less  than  a  year  of  programming  experience  and 
had  difficulties  with  features  of  the  programming  language 
rather  than  with  details  of  the  algorithms.  In  particular, 
in  our  sixth  experiment,  we  had  a  very  small  homogeneous 
group  of  subjects  and  consequently  achieved  extremely  small 
within  cells  variances  (typically  lower  than  on  experiments 
eight,  nine,  and  ten)  which  resulted  in  statistical 
significance  for  effects  too  small  to  be  detected  reliably 
in  other  experiments. 

Although  nothing  in  our  methodology  is  particular  to 
the  choice  of  subjects,  we  were  only  able  to  demonstrate  it 
with  students  as  subjects.  It  is  likely  that  most  future 
experimentation  will  be  done  with  relatively  unskilled 
(hence  cheap)  programmers.  As  Weinberg  [Weinberg  1971]  so 
aptly  put  it,  "Whereas  psychology  may  be  the  psychology  of 
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college  freshmen,  the 
become  the  psychology 


psychology  of  programming  could  easily 
of  programmer  trainees." 


3. 3  Capa! ili ties  and  lim it at  ions  of  the  Methodology 


Our  methodology  is  a  carefully 
a  collection  of  separately  good  featu 
inter-relationship  among  the  vari 
experiment.  The  contribution  made  by 
development  of  any  single  component 
rather  the  selection  of  suitabl 
combination  into  a  cohesive  unit,  and 
they  worked  well  together. 
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res.  There  is  a  strong 
ous  components  of  an 
this  thesis  is  not  the 
of  the  methodology,  but 
e  components,  their 
the  demonstration  that 


In  discussing  the  capabilities  of  the  methodology,  an 
important  question  that  arises  is  the  size  of  effect  that 
can  be  reliably  measured  in  a  given  experiment.  This  is  a 
function  of  both  the  number  of  subjects  and  the  variability 
of  subjects  (i.e.,  the  within  cells  variance). 
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evaluations,  the  within  cells  variances  ranged  from  1.08  to 
3.97.  On  the  quizzes,  the  within  cells  variances  ranged 
from  2.35  to  4.77.  On  the  modifications,  the  within  sells 
variances  ranged  from  1.83  to  3.52. 

We  can  now  estimate  the  number  of  subjects  needed  to 
measure  an  effect  of  a  given  size  using  these  within  cells 
variances.  In  our  analysis,  we  will  use  the  largest  within 
cells  variance  obtained  so  that  we  will  be  doing  a  worst- 
case  analysis.  The  results  of  this  analysis,  that  is,  an 
estimate  of  the  number  of  subjects  needed  to  significantly 
measure  at  <5%  level  an  absolute  difference  of  one  and  two 
units,  follows. 


Measure 


Seal  e 


Within  Cells 
Variance 


Estimate  of  Number 
_ of _ Subjects _ 

one  unit  two  units 


Self -Evaluation 

C-9 

3.  97 

80 

20 

Quiz 

0  —  1 C 

a.  77 

96 

24 

Modif ication 

0-5 

3.  52 

72 

18 
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that  we 

obtained 

in 

experiments  eigh 

t,  nine. 
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In  future 

e  xper ime 

nt  s , 

larger  within  cells  variances  may  occur,  in  which  case 
effects  of  the  size  discussed  above  would  not  be  significant 
with  the  given  number  of  subjects.  Conversely,  smaller 
within  cells  variances  may  occur  (as  in  our  sixth 
experiment) ,  in  which  case  effects  of  a  smaller  magnitude 
may  be  significant. 
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Our  general  rule  of  thumb  in  determining  the  size  of 
experiment  to  perform  (i.e.,  the  number  of  factors  and 
levels  to  study,  given  a  fixed  number  of  subjects)  is  to  try 
to  obtain  three  replicates  of  the  experiment.  We  did  this 
in  experiments  eight  and  ten,  but  only  had  two  replicates  in 
experiment  nine.  In  fact,  several  of  the  within  cells 
variances  used  in  the  above  worst-case  analysis  were 
obtained  in  experiment  nine,  our  two-replicate  experiment. 

There  are  also  statistical  techniques  that  can  be  used 

to  reduce  the  within  cells  variance  by  attributing  some  of 

the  variance  to  the  varying  abilities  of  subjects  as 

measured  by  such  things  as  pre-testing,  course  grades,  years 

of  experience,  etc.  These  techniques  include  least  squares 

fit  and  analysis  of  covariance  [Kirk  1968].  As  mentioned  in 

section  3.2.1  we  used  a  least  squares  fit  with  subjects' 

course  grades  on  our  second  experiment.  It  did  reduce  the 

within  cells  variance  slightly  (9.7%  on  the  hand-simulation 
« 

measure,  18.5%  one  the  fill-in-the  blanks  measure,  and  37.9% 
on  the  self-evaluation  measure) .  The  anonymity  of  subjects 
in  our  later  experiments  prevented  us  from  continuing  to  use 
such  techniques.  We  do,  however,  recommend  their  use 
whenever  possible. 

Another  important  question  that  arises  is  the  size  of 
the  typical  effect  that  is  likely  to  occur  in  an  experiment. 
This  a  function  of  many  things,  including  the  factor  being 
studied,  the  measure  being  used,  and  the  conditions  of  the 
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experiment.  A  table 

of 
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standard 

deviations  of 

the  differences  betwe 

en 

extreme 

levels 

of  the 

factors  that 

we  studied  in  experim 

ents  eight. 

nine. 

and  ten 

follows. 

Measure 

Self-Evaluation 

£uiz 

Modif ica tioa 

Control  Flow 

2. 20±. 

51 

.  60±. 14 

.  1 5±.  17 

Paragraphing 

1 .C6±. 

15 

1 . 25±.35 

.  85±. 49 

Factor 

Variables 

1 . 46±. 

33 

. 57±. 25 

. 39±. 21 

locality 

.  1 6±. 

1  2 

. 57±, 42 

. 36±. 28 

These  experiments  were  all  constrained  to  50  minutes. 
In  experiment  sixr  in  which  the  subjects  were  available  for 
two  hours,  some  of  the  effects  were  larger,  as  can  be  seen 
from  the  tables  in  Appendix  F. 

Finally,  we  will  discuss  some  limitations  of  our 
methodology.  One  limitation  is  the  size  of  effect  that  can 
be  measured  on  any  given  experiment.  This  has  already  been 
discussed  in  detail  and  is  a  function  of  the  measure,  the 
number  of  subjects,  and  the  within  cells  variance  of  an 
experiment . 

Another  limitation  is  that  it  is  not  reasonable  to 
believe  the  results  of  any  one  experiment,  no  matter  how 
statistically  significant  a  result  may  be.  In  fact.,  we 
recommend  conducting  two  or  more  experiments  with  different 
subjects  and  combinining  the  results,  rather  than  conducting 
one  large  experiment  with  additional  replicates.  This  is  to 
ensure  that  the  results  obtained  are  due  to  the  factors 
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being  varied  and  not  to  some  peculiarity  of  a  particular 
experimental  situation  (e.g.,  the  lighting  in  a  room). 

Another  limitation  is  that  there  may  be  components  of 
program  complexity  that  cannot  be  measured  by  such 
experiments.  In  the  next  chapter,  we  present  a  list  of 
factors  that  may  contribute  to  program  complexity  and  that 
can  be  measured  by  our  methodology.  We  do  not,  however, 
claim  that  this  list  is  complete  and  therefore  admit  the 
possibility  of  program  complexity  factors  for  which  our 
methodology  would  not  be  suitable. 

Another  limitation  is  that  the  very  nature  of 
controlled  experimentation  may  eliminate  interesting 
behaviour  that  occurs  in  a  normal  environment.  This  point 
is  emphasized  by  Weinberg  [1971]. 

finally,  our  methodology  is  limited  by  the  length  of 
experiment  that  we  can  hope  to  perform.  Two  hours  appears 
to  be  the  maximum  time  that  we  can  ex pect  for  programmer 
concentration  in  an  experiment  of  this  type.  Evidence  for 
this  comes  from  comments  made  by  subjects  themselves.  This 
limits  the  size  of  program  that  we  can  hope  to  study  using 

gy* 


our  methodolo 
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Chapter  4 

APPLICATION  OF  THE  METHODOLOGY 

This  chapter  presents  and  discusses  the  results  of 
experiments  that  we  performed.  It  contains,  therefore,  a 
demonstration  of  the  use  of  our  methodology. 

In  section  4.1,  we  present  our  list  of  complexity 
factors.  In  section  4.2,  we  discuss  the  results  obtained 
about  the  factors  that  we  studied. 


4.  1  Factors  Affecting.  Ik®  Complexity  of  Programs 


The  following  is  a  list  of  factors  that  we  thought 
might  contribute  to  the  complexity  of  programs.  The  list  is 
divided  into  four  sections.  Program  Form,  Control  Flow,  Data 
Flow,  and  Interaction  between  Control  and  Data  Flow.  The 
Program  Form  section  contains  those  factors  that  affect  only 
the  appearance  and  readability  of  programs.  The  remaining 
three  sections  contain  factors  that  also  affect  the  concent 
of  programs. 


The  list  was  arrived  at  from  our  own  personal 
experiences  and  opinions,  with  modifications  made  at  the 
suggestion  of  others.  Each  factor  is  followed  by  an 
explanation,  when  necessary,  and  an  intuitive  impression  of 
how  the  factor  should  affect  the  complexity  of  programs.  It 
is  the  purpose  of  experimental  studies  to  provide  evidence 
concerning  these  impressions. 
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Another  factor  that  surely  affects  the  complexity  o 
program  is  the  language  in  which  it  is  written.  The  rea 
that  it  is  not  in  the  list  is  that  it  affects  and 
affected  by  all  of  the  other  factors.  Consequently,  it 
not  clear  that  choice  of  language  can  be  systematica 
varied  independently  of  many  other  factors.  Thus, 
decided  to  hold  the  language  constant  in  each  experiu 
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to  use  two  different  languages.  In  any  given  experime 
however,  the  language  used  was  held  constant.  The 
languages  used  were  PI/I  and  ALGOL-W.  We  restric 
ourselves  to  simple  subsets  of  the  languages,  which  we  f 
would  introduce  relatively  little  variation  because  of  th 
similarity. 

4.1.1  Program  Form 

Presence  or  absence  of  extensive  comments 

We  feel  that  comments  can  increase  the  ability 
understand  and  maintain  programs.  It  should 
realized  however  that  meaningless  or  incorrect  comme 
can  be  harmful  [ Kernighan  and  Plauger  1974], 
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2.  Placement  of  declarations 

Declarations  of  variables  can  occur  at  various 
places  in  a  program  (e.g.,  at  the  beginning  of  their 
scope,  iust  prior  to  their  use,  after  their  use,  etc.)* 
Certain  placements  may  make  programs  more  difficult  to 
read  than  others  [Holt  1973,  Lecarme  and  Desjardines 
1974,  Weinberg  1971], 

3.  Paragraphing  of  program  listing 

Program  listings  that  are  indented  no  reflect  the 
structure  of  the  program  should  be  easier  to  read  than 
those  that  are  not  [Clark  and  Horning  1973]. 

4.  Choice  of  variable  names 

Mnemonic  variable  names  of  a  reasonable  length 
should  make  programs  more  understandable  than  short, 
non- mnemonic  variables  [Kernighan  and  Plauger  1974], 

c .  Redeclaration  of  a  variable  name  in  an  inner  scope 

We  feel  that  the  practice  of  using  the  same  name 
for  two  distinct  variables  increases  the  complexity  of 
a  program  since  it  may  be  confusing  which  variable  is 
actually  being  referenced,  especially  if  the 
declaration  is  poorly  placed  [Wulf  and  Shaw  19^3]. 


-60- 


6.  Use  of  "magic  numbers" 

"Magic  numbers"  are  numbers  whose  meanings  are  not 
implied  by  their  value.  For  example,  DO  1=1  TO  437  is 
perhaps  less  meaningful  than  DO  1=1  TO  TABLE_SIZE  where 
TABLE_SIZE  has  previously  been  equated  to  437  [Clark 
and  Horning  1973], 

4.1.2  Control  Flow 

1.  Complexity  of  control  flow  graph  of  the  program 

We  feel  that  various  properties  of  the  program's 
control  flow  graph  such  as  the  number  and  direction  of 
crossing  control  edges  may  affect  the  program's 
compl exitv. 


2.  Choice  of  control  constructs 

Constructs  such  as  the  GOTO  seem  to  encourage 
inscrutable  programs  whereas  contructs  such  as  IF  THEN, 
DO  WHILE,  and  CASE  seem  to  lead  to  more  understandable 
programs  [Diikstra  1968a,  Wulf  1971], 
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3.  Length  and  number  of  program  segments 

The  length  and  number  of  program  segments  (e.g., 
loops,  blocks)  may  affect  the  complexity  of  a  program. 
A  segment  that  is  long  may  be  too  difficult  to 
comprehend.  If  segments  are  too  short,  the  large 
number  of  them  required  may  increase  the  complexity  of 
the  program. 

4.  Passing  procedures  and  labels  as  parameters 

Although  a  useful  technique,  particularly  in  some 
numerical  applications,  it  can  often  make  the  flow  of 
control  too  confusing  to  follow.  This  is  especially 
true  in  the  case  of  labels. 

5.  Recursion 

The  use  of  recursive  procedures  can  often 
simplify  the  specification  of  an  algorithm,  although 
following  the  flow  of  execution  through  various  levels 
of  recursion  may  become  more  difficult. 

6.  Levels  of  nesting 

Various  constructs  (e.g.,  blocks,  IF  THEN'S, 
iterative  constructs)  can  often  be  nested  to  arbitrary 
depths.  If  the  nesting  is  too  deep,  however,  the 
program  may  become  unintelligible. 


-62- 


4.1.3  Data  Flow 

1  .  Scope  of  variables 

The  range  of  program  over  which  variables  have 
scope  (i.e. ,  global  vs.  local)  may  affect  the  ability 
to  understand  and  modify  a  program.  Global  variables 
that  are  automatically  inherited  in  inner  scopes  may  be 
harmful  [Wulf  and  Shaw  1973], 

2.  Clustering  of  data  references 

Programs  in  which  each  variable  is  referenced  in  a 
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[  Naur 

1969, 

McKeeman 

1972c  ]. 

3.  Declaration  and  use  of  data  structures 

Defining  new  data  structures  for  a  particular 
application  may  help  to  make  a  program  more 
understandable. 

4.  Locality  of  operations  performed  on  data  structures 

As  in  the  case  of  simple  variables,  programs  in 
which  data  structures  are  referenced  in  a  restricted 
locale  should  be  easier  to  understand  and  modify 
[Liskov  and  7illies  1974  ]. 


-63- 


5.  Use  of  pointers 

Programs  containing  pointers  are  felt  to  be  more 
complex  than  those  without.  This  is  especially  true  if 
the  pointers  are  not  restricted  to  point  to  a 
particular  type  of  variable  or  if  the  pointers  can 
potentially  point  to  variables  that  no  longer  exist 
[ Hoare  1973,  Kieburtz  1973]. 

6.  Arithmetic  on  pointers 

Programs  that  contain  arithmetic  operations 
performed  on  pointers  seem  to  be  more  complex. 

4.1.4  Interaction  between  Control  and  Data  Flow 

1.  Flag  testing  (in  absence  of  EXIT  construct) 

Often  it  is  desired  to  terminate  processing  of  a 
loop  somewhere  in  the  middle.  This  can  be  accomplished 
with  a  GOTO  or  "XIT  construct.  If,  however,  we  are  not 
allowing  GOTO's  and  our  language  does  not  contain  an 
EXIT,  flags  must  be  introduced  and  tesred  to  avoid 
executing  various  portions  of  the  loop.  This  can  make 
rhe  program  more  difficult  to  understand  [Hopkins  1972, 
Knuth  1974  ]. 
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2.  Side  effects  affecting  control  flow 

Side  effects  (e.g.,  a  procedure  changing  the  value 
of  a  global  variable)  that  later  affect  the  control 
flow  of  a  program  can  be  difficult  to  comprehend  [Knuth 
1067  ]. 


3. 

Changing  the  iteration 

variable 

Loops  in  which 

the  iteration 

variable  is 

changed 

internally  are  often 

difficult  to 

understand. 

This 

practice  is  frequently  used  to  cause  early  termination 
of  a  loop  [Knuth  1967], 

4.  Method  of  parameter  passing 

I 

The  method  of  parameter  passing  (e.g.,  by  name,  by 
value,  by  reference,  by  result,  by  value  result)  may 
affect  the  ease  with  which  the  program  can  be 
understood.  Parameters  passed  by  value  and  by  result, 
although  not  as  powerful  as  the  other  methods,  may  make 
programs  easier  to  understand. 

4  •  2  Z§.2i2£§  Studied 

In  this  section  we  will  discuss  the  results  of  our 
experiments,  factor  by  factor.  When  an  interaction  occurred 
between  factors,  it  will  be  discussed  in  the  sections 
pertaining  to  the  factors  involved  in  the  interaction.  The 
results  of  the  statistical  analysis  of  the  data  are 


-6  5- 


contained  in  Tables  1  -  38  (in  Appendices  A  -  J)  ,  but 
particularly  interesting  data  will  be  duplicated  here.  The 
data  presented  in  the  tables  and  throughout  this  section  are 
mean  scores  for  the  various  levels  of  the  factors.  When  we 
discuss  a  "significant”  difference  between  levels  of  a 
factor,  we  mean  an  effect  that  is  statistically  significant 
at  <5f  level  unless  otherwise  specified. 

As  can  be  seen  throughout  the  tables,  the  main  effect 
associated  with  choice  of  program  is  often  statistically 
significant.  This  may  indicate  either  that  one  program  is 
mere  difficult  than  the  other  or  that  one  quiz  or 
modification  is  more  difficult  than  the  other. 
Consequently,  a  statistically  significant  difference  between 
programs  is  not  a  very  interesting  result.  What  is 
interesting,  and  important,  is  the  occurrence  of  only  two 
significant  interactions  among  choice  of  program  and  the 
other  factors  in  all  of  our  experiments.  This  indicates 
that  the  other  effects  we  observed  are  not,  in  most  cases, 
sensitive  to  the  choice  of  program.  Also,  out  of  22 
significant  effects,  only  four  significant  interactions 
occurred  among  the  other  factors  (and  two  of  these  were  on 
hand- simulation)  .  This  supports  our  assumption  that  we  can 
study  psychological  complexity  by  individually  studying 


factors  that  contribute  to  it 
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4,2.1  In  Programs 
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act  as  somewhat  of  a  stabilizing  influence  to  counteract  xhe 
error-producing  effect  of  comments  on  hand-simulation. 

.A  somewhat  counter-intuitive  significant  interaction 
(at  <2.5%  level)  occurred  between  comments  and  paragraphing 
on  the  number  of  correct  values  hand-simulated.  When  the 
listing  was  not  paragraphed,  the  presence  of  comments 
overwhelmingly  helped  performance:  44.2  values  vs.  26.2 


values ;  but 
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was  paragraphed. 

the  prese 

nee 

of  comments 

hurt 

perf orma nee 

slightly:  23.7 

values  vs.  2 

9.7 

values.  We 

will 

discuss  this 

interaction  in 

the  seexion 

on 

paragraphing. 

On  the  f ill-in-t he-blanks  measure  of  understanding  that 
we  used  in  this  experiment,  the  presence  of  comments 
slightly  helped  performance:  6.0  vs.  5.1  but  this  result 
was  not  statistically  significant. 

On  the  self-evaluation  we  again  achieved  statistically 
significant  results  with  regard  to  comments.  The  main 
effect  of  comments  was  mildly  significant  (<1C%  level)  and 
suggested  that  comments  had  helped  performance  (4.0  vs. 
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Overall,  this  experiment  supports  the  usefulness 
extensive  comments.  On  both  measures  of  understandi 
f ill- in-t he-blanks  and  self-evaluation,  comments  impro 
performance,  although  interacting  with  paragraphing  on  ss 
evaluation.  On  han d- simulation ,  comments  did  produce  m 
errors,  but,  as  mentioned  in  Chapter  3,  we  do  not  consi 
hand- simu laTion  a  measure  of  understanding. 

Clearly,  further  investigation  with  more  levels 
comments  is  needed  to  determine  more  precisely  w 
constitutes  a  meaningful,  well-placed  comment.  The  limi 
number  of  experiments  that  we  were  able  to  conduct  preven 
us  from  studying  this  factor  further. 

4.2.2  Control  Flow 

We  studied  two  factors  from  the  Control  Flow  secti 
complexity  of  the  control  flow  graph  of  the  program  and 
choice  of  control  constructs.  A  problem  that  we  encounta 
in  trying  to  vary  the  first  factor  consisted  of  deterrain 
what  constitutes  a  complex  control  graph.  An  ove 
simplistic  view  is  that  planar  graphs  are  simple  and  t 
ncn-planar  graphs  are  complex.  The  problem  with  this  v 
comes  from  the  fact  that  a  graph  can  be  imbedded 
different  ways  (e. g. ,  a  planar  graph  can  have  a  non-pla 
imbedding).  Although  planarity  as  such  does  not  daterra 
the  complexity  of  a  control  graph,  we  do  feel  that  cross 
edges  make  a  control  graph  more  complex.  In  fact,  we  f 
that  edges  crossing  in  opposite  directions  are  more  comp 


of 

ng, 

ved 

lf- 

ore 

der 


of 

hat 

ted 

ted 


on : 
the 
red 
ing 
r  ly 
hat 
iew 
in 
nar 
ine 
ing 
eel 
lex 


-69- 


than  edges  crossing  in  the  same  direction.  However,  we  used 
only  two  extreme  levels  of  this  factor.  At  one  level  we  had 
a  "simple"  control  graph  with  few  crossing  edges  and  only 
crossing  in  the  same  direction.  At  the  other  level  we  had  a 
"complex"  control  graph,  with  many  crossing  edges  and 
crossing  both  in  the  same  and  opposite  directions. 


We  also  used  two  extreme  levels  of  the  second  factor, 
choice  of  control  constructs.  At  one  level  we  had  only 
"structured"  constructs,  IF_THEN_ELSE ,  DO  WHILE,  etc.  At 
the  second  level  we  used  only  GOTO' s.  Here  we  again  ran 
into  a  problem.  We  discovered  that  we  could  not  vary  these 
two  factors  independently  because  we  could  not  producs  a 
"complex"  control  graph  using  only  "structured"  control 
constructs.  Therefore,  we  combined  these  two  factors  into 


one,  with  three 

levels: 

1) 

"structured " 

constructs. 

"simple"  control 

flow ; 

2) 

" un  structured" 

construe  ts. 

"simple"  control 

flow; 

3) 

"unstructur  ed" 

constructs. 

"complex"  control  flow. 


We  studied  this  compound  factor  in  experiments  three 
and  nine.  As  described  in  Chapter  3,  experiment  three  did 
not  work  out  very  well  because  of  the  new  procedure  of 
variable  time  limits  that  we  were  trying.  We  obtained 
incomplete  data  and  consequently  nothing  significant 
appeared. 

In  experiment  nine,  we  studied  three  levels  of  control 
flow  and  two  levels  of  paragraphing.  We  achieved 


0- 

stat i stically  significant  results  with  regard  to  control 
flow  on  all  three  self-evaluations.  On  the  first  self- 
evaluation  the  means  for  the  three  levels  of  control  flow 
were:  3.1,  2.0,  1.5.  This  effect  was  significant  at  <5% 
level.  Comparing  these  differences  pair-wise,  we  find  that 
the  difference  between  the  first  and  second  levels  is 
significant  at  <10%  level,  the  difference  between  the  first 
and  third  levels  is  significant  at  <2.5%  level,  and  the 
difference  between  the  second  and  third  levels  is  not 
significant. 


On 

the  second 

sel f-eva luation. 

similar  results 

appea  red . 

The  means  fo 

r  the  three  levels 

of  control  flow 

were: 

1 ,  2.0,  1.6 

.  This  effect  was 

significant  at  <5% 

level . 

Considering 

the  differences 

pair-wise,  the 

difference  between  the  first  and  second  levels  is 
significant  at  <10%  level,  the  difference  between  the  first 


third 

level s 

is  significant 

at  <5% 

level,  and 

the 

e  rence 

between 

the  second  and 

third 

levels  is 

not 

s igni f icant . 

On  the  third  self-evaluation,  the  means  for  the  three 
levels  of  control  flow  were:  4.5,  2.6,  2.0.  This  effect 
was  significant  at  <5%  level.  The  difference  between  the 
first  and  third  levels  was  significant  at  <2.5%  level,  and 
the  differences  between  the  first  and  second,  and  second  and 
third  levels  were  not  significant. 
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On  -the  first  modification  that  the  subjects  made  to  the 
program,  the  means  for  the  three  levels  of  control  flow 
were:  1.4,  2.C,  1.3.  This  effect  was  not  significant. 
What  was  significant,  however,  was  the  interaction  between 
control  flow  and  paragraphing.  This  is  discussed  in  detail 
in  the  section  on  paragraphing.  What  should  be  noted  here, 
however,  is  that  subjects  at  level  one  of  control  flow  with 
paragraphed  listings  performed  better  than  subjects  in  any 
other  cell:  2.8  vs.  0,  1.9,  2.1,  0.8,  1.8. 

On  the  second  modification  the  means  were:  3.1,  2.8, 
2.9.  This  effect  was  not  significant. 

On  the  first  quiz,  the  means  were:  3.9,  2.9,  3.4,  and 
on  the  second  quiz,  the  means  were:  4.6,  3.6,  3.9.  These 
effects  were  not  significant. 

While  no  definite  conclusions  can  be  drawn  from  this 
experiment,  certain  observations  can  be  made.  Subjects  with 
programs  at  level  one  of  control  flow  ("structured" 
constructs,  "simple"  control  flow)  performed  better  on  all 
measures  of  performance  than  subjects  at  the  other  two 
levels,  except  on  the  first  modification.  On  the  first 
modification,  however,  the  interaction  occurred  with 
paragraphing  and  subjects  with  paragraphed  programs  at  level 
one  of  control  flow  performed  better  than  subjects  in  any  of 
the  other  cells.  On  all  three  self-evaluations,  the 
differences  between  the  first  and  third  levels  of  control 
flow  was  significant  and  on  the  first  and  second  self- 


evaluations  the 


differences  between  the  first  and  second 


levels  of  control  flow  were  mildly  significant  (<10%  level). 
This  would  tend  to  support  the  usefulness  of  "structured" 
control  constructs. 

Comparing  the  second  and  third  levels,  however,  the 
evidence  is  less  conclusive.  On  the  three  self-evaluations 
and  the  first  modification,  subjects  performed  better  at  the 
second  level  of  control  flow  than  at  the  third.  On  the 
second  modification  and  on  both  quizzes,  subjects  performed 
better  at  the  third  level.  These  effects  were  not 
significant.  This  tends  to  indicate  that  as  long  as 
"unstructured"  constructs  (i.e.,  GOTO’s)  occur  in  a  program, 
it  makes  relatively  little  difference  whether  the  control 
flow  is  "simple"  or  "complex".  Before  coming  to  this 
conclusion,  however,  it  should  be  noted  that  the  subjects 
used  in  this  experiment  had  been  previously  trained  with  the 
ideas  of  structured  programming  and  taught  that  GOTO 
statements  are  "bad".  It  would  be  interesting,  and 
important,  to  repeat  this  experiment  with  subjects  that  had 
not  been  so  trained,  e.g.,  FORTRAN  programmers. 


4.2.3  Paragraphing 

^7e  studied  the  effect  of  paragraphing  the  program 
listing  in  experiments  one,  two,  and  nine.  Experiment  one 
was  a  pilot  study  and  did  not  achieve  any  statistically 
significant  results.  In  both  the  second  and  ninth 
experiments,  we  obtained  statistically  significant  results. 
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Because  our  measures  of  understanding  changed  substantially 
between  these  experiments,  we  cannot  meaningfully  combine 
the  results  and  will  therefore  discuss  them  separately. 

In  both  experiments  we  studied  paragraphing  at  two 
levels.  At  one  level,  the  program  text  was  indented  to 
reflect  the  structure.  At  the  other  level,  there  was  no 
indentation. 

In  experiment  two,  the  main  effect  of  paragraphing  was 
not  significant  on  any  of  our  measures,  but  a  significant 
interaction  (at  <2. 5*  level)  occurred  with  comments  on  two 
of  them.  The  first  interaction  involved  the  number  of 
correct  values  hand-simulated.  When  comments  were  absent, 
paragraphing  helped  slightly:  29.7  values  vs.  26.2  values; 
but  when  comments  were  present,  paragraphing  hurt 
drastically:  23.7  values  vs.  44.2  values.  The  second 
interaction  (at  <.1%  level)  involved  the  self-evaluation. 
When  comments  were  absent,  paragraphing  helped:  4.2  vs. 
2.1;  but  when  comments  were  present,  paragraphing  hurt:  3.4 
vs.  4.7. 

This  negative  effect  of  paragraphing  is  quite 
surprising.  While  we  cannot  offer  a  definitive  explanation 
based  on  this  experiment,  we  can  offer  a  hypothesis.  The 
programs  used  in  this  experiment  did  not  fit  on  a  single 
cage  and  had  to  be  split  across  page  boundaries.  We  were 
not  very  careful  in  this  experiment  and  did  not  split  the 
programs  at  reasonable  points.  In  addition,  the  programs 
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contained  GOTO  statements,  which  do  not  lend  themselves  to 
paragraphing.  While  this  may  partially  explain  the  negative 
effect  of  paragraphing,  it  does  not  explain  the  interaction 
with  comments.  Further  experimentation  is  required  here. 
Although  we  did  later  experiment  with  paragraphing  in 
experiment  nine,  we  were  not  able  to  study  further  its 
interaction  with  comments. 


In  experiment  nine,  we  again  studied  the  same  two 
levels  of  paragraphing,  but  this  time  in  connection  with 
three  levels  of  control  flow.  We  had  a  problem  in 
determining  what  it  meant  for  the  paragraphing  to  reflect 
the  structure  of  a  program  at  the  third  level  of  control 
flow  (’’unstructured"  constructs,  "complex"  control  flow)  , 
which  effectively  has  no  structure.  We  made  the  arbitrary 
decision  that  since  a  program  at  level  three  of  control  flow 
was  constructed  by  randomizing  the  sections  of  the  program 
at  level  two,  then  a  "paragraphed"  program  at  level  three  of 
control  flow  would  have  the  statements  indented  the  same 
number  of  spaces  that  they  were  in  level  two.  We  felt  that 
this  might  be  meaningless  and  included  it  only  to  complete 
the  factorial  design.  In  fact,  we  expected  that  there  might 
be  an  interaction  between  control  flow  and  paragraphing  such 
that  the  paragraphing  would  aid  performance  at  the  first  two 
levels  of  control  flow  and  hinder  it  slightly  at  the  third 
level.  As  will  be  discussed  below,  an  interaction  almost  of 
this  type  did  occur  significantly,  but  only  once,  on  the 
first  modification  made  to  the  program.  On  all  other 
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measures,  paragraphing  aided  performance,  sometimes 
significantly,  at  all  levels  of  control  flow  including  the 
third.  The  only  exceptions  to  this  were  the  two  quizzes,  on 
which  paragraphing  hurt  very  slightly  at  the  second  level  of 
control  flow,  but  these  differences  were  not  significant. 

has  already  been  indicated,  the  presence  of  a 
paragraphed  listing  appears  to  have  helped  on  all  of  our 
measures.  On  the  first  self-evaluation,  the  main  effect  of 
paragraphing  was  statistically  significant  at  <5%  level: 

2.8  vs.  1.7.  On  the  second  and  third  self-evaluations, 
paragraphing  also  helped  although  not  significantly:  3.2 
vs.  2.0  on  the  second  and  3.5  vs.  2.6  on  the  third.  On  the 
first  quiz,  paragraphing  helped  although  not  significantly: 

3.9  vs.  2.9,  On  the  second  quiz  the  main  effect  of 
paragraphing  was  mildly  significant  (<10%  level):  4.8  vs. 
3.3. 

The  above-mentioned  interaction  involving  paragraphing 
appeared  on  the  first  modification.  The  interaction  with 
all  three  levels  of  control  flow  is  only  mildly  significanr 
(<1C%  level).  f.t  the  first  level  of  control  flow, 
paragraphing  helped  performance:  2.8  vs.  0,  at  the  second 
level,  paragraphing  hurt  slightly:  1.9  vs.  2.1,  and  at  the 
third  level,  paragraphing  hurt  performance:  0.8  vs.  1.8.  A.s 
mentioned  before,  we  do  not  consider  this  surprising  in  view 
of  the  meaninglessness  of  paragraphing  at  the  third  level  of 


control  flow. 
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The  results  of  this  experiment  tend  to  support  the 
usefulness  of  paragraphing,  since  in  most  cases  it  improved 
performance,  in  some  cases  it  significantly  improved 
performance,  and  in  no  meaningful  cases  did  it  significantly 
hurt  performance. 


further  experimentation  is  clearly  possible  with  other 
levels  of  paragraphing.  Many  people  have  their  own  styles 
of  paragraphing  program  listings  and  these  can  be  compared 
against  one  another  in  experiments.  Unless  the  subjects' 
backgrounds  are  controlled,  however,  the  results  of  such  an 
experiment  may  not  determine  which  style  of  paragraphing  is 
better,  but  rather  which  style  the  subjects  are  used  to. 


4.2.4  Variable  Names 


We  studied  the  effect  of  choice  of  variable  names  in 
experiments  one,  two,  six,  eight,  and  ten.  Experiment  one 
was  a  pilot  study  and  did  not  achieve  any  statistically 
significant  results.  Experiments  six,  eight,  and  ten  were 


similar  enough  nature  that  we  will  discuss  th 


em 


together.  Experiment  two  was  different  from  these 
experiments  in  the  procedures  used,  the  measures  used,  and 
the  levels  of  the  factor  studied,  so  we  will  discuss  it 
separately. 


In  experiment  two,  we  studied  variable  names  at  two 
levels.  At  one  level  we  had  mnemonic  variable  names,  that 
is,  variable  names  that  were  intended  to  be  meaningful  and 
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easily  remembered,  and  at  the  other  level  we  had  short, 
meaningless  (one-  and  two-letter)  names. 

One  effect  of  variable  names  in  this  experiment 
appeared  on  hand-simulation.  Subjects  with  programs 
containing  mnemonic  variables  hand-simulated  significantly 
less  rapidly  than  subjects  with  programs  with  short  variable 
names:  37.7  values  vs.  48.5  values.  In  addition,  subjects 
with  programs  with  mnemonic  variables  made  significantly  (a z 
<1%  level)  fewer  errors  than  those  without:  8.3  vs.  15. C. 
In  this  case,  however,  a  significant  interaction  occurred 
with  comments.  When  comments  were  present  and  mnemonic 
variables  were  absent,  a  significantly  larger  number  of 
errors  were  made  than  in  any  of  the  other  cases:  22.0  vs. 
10.  C,  8.  C,  and  8.7.  This  interaction  was  discussed  in  the 
section  on  comments. 

On  the  f ill-in-t he-blanks  measure  of  understanding  that 
we  used  in  this  experiment,  mnemonic  variables  slightly 
helped  performance:  5.7  vs.  5.3.  Similarly  on  the  self- 
evaluation,  mnemonic  variables  helped  slightly:  3.8  vs. 
3.4.  Neither  of  these  results,  however,  was  statistically 
significant. 


In  experiment  six,  we  studied  variable  names  at  two 
levels.  At  one  level  we  had  mnemonic  variable  names  and  at 
the  other  level  we  had  shortened  mnemonics  which  were 
constructed  by  deleting  all  vowels  from  the  mnemonic  name 
except  the  first  letter  if  it  happened  to  be  a  vowel.  This 
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was  an  attempt  to  keep  the  meaning  of  the  name  reasonably 
constant  while  changing  its  length  and  pronounceabilit y.  In 
experiments  eight  and  ten  we  studied  these  same  two  levels 
and  also  added  a  third  level.  -At  this  level  we  had  names 
that  had  the  same  number  of  letters  and  syllables  as  the 
mnemonic  names  but  whose  meaning  bore  no  relation  to  the 
program.  In  these  two  experiments,  we  also  studied  an 
additional  factor,  locality  of  data  references. 


Experiments  eight  and  ten  were  almost  identical  except 
for  a  slight  procedural  difference  and  the  use  of  different 
subjects.  Consequently,  they  can  be  combined  directly. 
Experiment  six  can  be  combined  with  experiments  eight  and 
ten  by  considering  only  the  first  two  levels  of  variable 
names  in  these  latter  two  experiments.  Also,  because  of  the 
procedural  differences  in  these  experiments,  the  first  and 
second  quizzes  of  experiments  eight  and  ten  are  actually 
comparable  to  the  second  and  third  quizzes,  respectively,  of 
experiment  six. 


In  combining  the  res 
ten,  we  used  the  meth 
distributions  as  discus 
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In  experiment  six,  mnemonic  variable  names  were 
significantly  better  than  shortened  mnemonics  on  all  three 
self-evaluations.  On  the  first  self-evaluation,  the 
difference  was  significant  at  <5%  level:  5.5  vs.  4.0 
although  a  significant  interaction  did  occur  with  choice  of 
program.  On  the  second  self-evaluation,  the  difference  was 
significant  at  <1%  level:  7.25  vs.  3.5.  On  the  third  self- 
evaluation  the  difference  was  significant  at  <2.5%  level: 
7.75  vs.  3.75.  On  the  three  quizzes,  the  differences  *re re 
not  significant,  but  mnemonic  variables  were  better  than 
shortened  mnemonics  on  all  but  the  first  quiz. 


This  experiment  taken  by  itself  might  tend  to  indie 
that  mnemonic  variables  are  far  superior  to  shorte 
mnemonics,  but  this  result  was  not  significantly  confir 
on  either  experiment  eight  or  ten.  Using  the  chi-sgj 
method  to  combine  experiments  six,  eight,  and  ten  indioa 
that  the  differences  between  mnemonics  and  shorts 
mnemonics  on  the  three  self-evaluations  are  not  significa 


Thus,  experiments 

eight  and 

ten  tend 

to  reduce 

significance  of 

exp  sr iment 

six.  In 

fact,  the 

significant  difference  that  occurred  between  the  first 
levels  of  variable  names  on  these  two  experiments  was  on 
first  modification  on  experiment  eight  in  which  mnemo 
variables  were  significantly  better  than  shortened  mnemon 
at  <5%  level:  3.3  vs.  1.8.  This  did  not  reapp 

significantly  in  experiment  ten,  and  the  combined  result 
not  significant.  In  the  version  of  one  of  the  programs 
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experiment  eight,  and  ten  with  subroutines  and  shortened 
mnemonics,  we  accidentally  failed  to  shorten  three  of  the 
mnemonic  variable  names.  This  may  partially  account  for  the 
reduced  effect  on  these  two  experiments. 

The  third  level  of  variable  names  that  we  added  in 
experiments  eight  and  ten  produced  many  significant  results. 
On  all  three  self-evaluations,  both  mnemonic  and  shortened 
mnemonic  variables  were  significantly  better  than  the 
meaningless  but  pronounceable  variable  names  in  both 
experiments.  On  the  first  self-evaluation  the  means  for  the 
three  levels  of  variable  names  were  2.1,  2.4,  0.8  on 
experiment  eight,  and  2.8,  3.1,  1.3  on  experiment  ten. 
Combining  the  two  experiments,  the  mnemonics  were 
significantly  better  than  the  meaningless  variables  at  <.  1% 
level  and  the  shortened  mnemonics  were  significantly  batter 
rhan  the  meaningless  variables  at  <.05%  level. 


On  the  second  self-evaluation,  the 
levels  of  variable  names  were:  2.6,  2.5, 
eight  and  4.0,  3.8,  2.0  on  experiment 

two  experiments,  mnemonic  variables  w 
better  than  meaningless  variables  at 
shortened  mnemonics  were  better  than 
variables  at  <5%  level. 


means  for  the  three 
1.8  on  experiment 
ten.  Combining  the 
ere  significantly 
<. 5%  level  and  the 
the  meaningless 


On  the  third  self-evaluation,  the 
levels  of  variable  names  were  3.6,  3.2, 
eight  and  3.6,  3.1,  2.  2  on  experiment 


means  for  the  three 
2.C  on  experiment 
ten.  Combining  the 


-81- 


two  experiments,  mnemonic  variables  were  si 
better  than  meaningless  variables  at  <5%  level  an 
mnemonics  were  significantly  better  than 
variables  at  <10%  level.  It  is  interesting  to  no 
difference  between  mnemonic  and  shortened 
variables,  although  not  significant,  continued 
from  one  self-evaluation  to  the  next  on  both  expe 


gnif ican tly 
d  shortened 
meaningless 
te  that  the 
mnemonic 
to  increase 
riments. 


On  the  quizzes  and  modifications,  only  two  significant 
effects  concerning  variable  names  appeared.  The  first  was 
the  significant  difference  between  mnemonic  variables  and 
shortened  mnemonics  on  the  first  modification  in  experiment 
eight  that  was  mentioned  above.  The  other  was  a  significant 
interaction  between  variable  names  and  locality  of  data 
references  on  the  second  modification  in  experiment  eight. 
The  means  for  this  interaction  appear  in  Table  19.  Since  it 
did  not  occur  in  experiment  ten,  and  since  we  have  no 
suitable  hypothesis  to  explain  it,  we  will  not  discuss  it 
further. 


The  other  effects 
modifications  concernin 
statistically  significan 
however,  there  is  a  signi 
of  the  effects  on  the 
followed  them,  and  on  the 
general.  There  is  not 


that  occurred  on  the  quizzes 
g  variable  names  are 
t.  As  mentioned  in  Chapter 
ficant  agreement  in  the  direct 
modifications  and  the  quizzes  r 
quizzes  and  self-evaluations 
such  an  agreement,  however,  betw 
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the  modifications  and  the  self-evaluations  that  followed 
them. 


The 

indications 

of  these 

experim 

ent  s 

are  that 

both 

m  nemonic 

and 

shortened 

mnemonic 

variabl es 

are 

b  etter 

than 

meaningl 

e  ss 

var iabl es 

,  and 

mnemonic 

variables  may 

be 

slightly  better  than  shortened  mnemonics,  as  measured  by  a 
programmer's  perception  of  how  well  he  feels  he  understands 
a  program.  Fxperiment  six  is  one  of  the  rare  cases  in  which 
a  sianificant  interaction  with  program  occurred,  and  the 
relationship  between  mnemonic  variables  and  the  complexity 
of  particular  programs  should  be  investigated  further. 

Even  though  the  effect  of  variable  names  does  not 
appear  strongly  on  the  objective  measures,  we  should  not 
overlook  its  importance  on  the  subjective  measure.  If 
programmers  feel  that  they  understand  one  form  of  a  program 
better  than  another,  then  we  should  strive  to  produce 
programs  of  that  form,  not  only  to  keep  them  happy,  but  also 
to  reduce  the  amount  of  effort  that  they  feel  they  must  put 
in  to  understand  and  modify  the  program. 

4.2.  5  Locality  of  Fata  References 

In  experiments  four,  eight,  and  ten,  we  studied  a 
combination  of  factors  from  the  Data  Flow  section  of  our 
list:  the  locality  of  references  to  both  simple  variables 
and  data  structures.  In  all  three  experiments,  we 
considered  two  levels  of  this  factor.  At  one  level,  we  used 
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a  straight-line  program  in  which  nc  attempt  was  made  to 
localize  the  data  references.  At  the  other  level,  we  used 
subroutines  to  localize  the  data  references  by  abstracting 
the  operations  to  be  performed  on  the  data.  This  is  not  the 
only  possible  method  of  localizing  data  references,  but 
seemed  reasonable  in  the  absence  of  a  language  that  allows 
user-defined  data  types.  As  a  side  comment,  we  found  PL/I 
procedures  with  multiple  entry  points  superior  for  this 
purpose  than  ALGOL-W  procedures. 


In  experiment  four  no  significant  results  appeared, 
the  time  we  felt  that  this  was  due  to  the  fact  that 
programs  were  too  easy  for  the  subjects. 

In  experiment  eight,  the  only  significant  effect  t 
appeared  concerning  locality  of  data  references  was 
interaction  with  choice  of  variable  names  on  the  sec 
modification,  which  we  are  at  a  loss  to  explain. 

In  experiment  ten,  the  only  significant  effect  t 
appeared  concerning  locality  of  data  references  occurred 
the  first  quiz.  Subjects  with  straight-line  progr 
performed  better  than  subjects  with  programs  w 
subroutines:  2.2  vs.  1.3.  This  effect  was  only  signific 

at  <10%  level.  However,  if  the  programs  with  meaningl 
variable  names  are  removed,  subjects  with  straight-1 
programs  performed  significantly  better  (<5%  level)  t 
subjects  with  programs  with  subroutines:  2.6  vs.  1.0. 
this  experiment  is  combined  with  experiment  eight  using 
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chi-square  distribution  method,  however,  this  difference  is 
no  longer  significant. 

We  were  originally  surprised  by  the  fact  that  very 
little  effect  was  caused  by  the  locality  of  data  references 
and  especially  by  the  fact  that  the  one  significant  effect 
was  not  in  the  direction  we  expected.  Intuitively,  we  feel 
that  localizing  the  operations  performed  on  data  should  make 
programs  easier  to  understand  and  modify.  What  we  realized, 
however,  was  that  the  programs  that  we  used  in  the 
experiments  were  fairly  short  and  that  the  straight-line 
versions  probably  were  somewhat  simpler  than  the  subroutined 
versions.  The  positive  effect  of  localizing  the  data 
references  in  a  program  would  probably  not  show  up  until  the 
program  became  fairly  large. 


One  factor  affecting  the  com 
sheer  size.  Consequently  any  re 
programs  may  or  may  not  hold 
further  experimentation  will  dete 
it  is  harder  and  more  "ex pen 
experiments  using  large  progra 
involved  and  the  patience  of  the 


plexity  of  a  program  is  its 


suits 

obtained 

for 

small 

for 

large  programs. 

Dnly 

rm  ine 

that.  Unf ortunata iy, 

si  ve" 

to  perform 

controlled 

ms  because  of 
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time 

subjects. 

4 . 3  f ummary  of  Results 

In  this  section  we  will 
about  the  various  factors  that 


briefly  summarize  the  results 
we  studied,  as  discussed  in 
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the  previous  sections.  We  cannot  draw  definite  conclusi 
about  any  of  the  factors  because  of  the  limited  number 
experiments  performed  and  the  similarity  in  the  backgro 
of  the  subjects  used. 


Extensive  comments  were  only  studied  in  an  ea 
experiment.  They  did,  however,  aid  performance,  except 
hand- simulati on.  The  improved  performance,  however,  was 
stat istically  significant,  although  the  effect  on  ha 
simulation  was. 


Paragraphing  aided  performance  in  most  cas 
significantly  in  some  cases,  and  never  significantly  4 
performance  except  in  an  interaction  with  comments.  On 
measures  in  which  significant  differences  occurr 
paragraphing  aided  performance  by  45%  -  85%.  This  w 

however,  our  own  particular  style  of  paragraphing. 

Subjects  performed  better  with  programs  contain 
"structured"  control  constructs  than  with  progr 
containing  GOTO  statements.  "Structured"  constructs  w 
significantly  better  than  GOTO's  with  "simple"  control  fi 
improving  self-evaluations  by  55%  -  105%.  They  w 

significantly  better  than  GOTOfs  with  "complex"  cont 
flow,  improving  self-evaluations  by  105%  -  155%,  W 

GOTO's  were  present  in  programs,  it  is  less  clear  than 
complexity  of  the  control  flow  affects  performa 
significantly.  This  result,  in  particular,  may  be  affec 
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Both  mnemonic  and  shortened  mnemonic  variable  names 
were  better  than  meaningless  variable  names,  and  mnemonic 
variables  were  slightly  better  than  shortened  mnemonics.  On 
the  self-evaluations;  mnemonic  variables  were  significantly 
better  than  meaningless  variables  by  45%  -  160%;  shortened 
mnemonics  were  significantly  better  than  meaningless 
variables  by  40%  -  200%;  and  on  one  experiment  out  of  three, 
mnemonic  variables  were  significantly  better  than  shortened 
mnemonics  by  35%  -  105%. 

Localizing  data  references  by  using  subroutines  had 
very  little  effect  and  in  a  negative  direction.  This,  we 
feel,  was  due  to  the  fact  that  the  programs  used  were  fairly 

short . 

A  tabulation  of  the  statistically  significant  results 
obtained  in  our  experiments  follows. 


factor  (s)  Significance  Table 


Bxp. 

Involved 

Measure 

Level 

Number 

2 

paragraphing 

comments 

correct 

hand- simulation 

<2.  5% 

2 

2 

comments 

errors  in 
hand- simulation 

<5% 

3 

2 

variables 

errors  in 
hand-simu la t ion 

<1% 

3 

2 

comments 

variables 

errors  in 
hand- simulation 

<5% 

3 

2 

comments 

total 

hand- si mu lat ion 

<2.  5% 

4 

2 

variables 

total 

hand- simulation 

<5% 

4 

2 

6 

6 

6 

8 

8 

8 

8 

8 

9 

9 

9 

g 

10 

10 

1  0 
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Factor  (s) 
Involved 

Measure 

Significance 

Level 

Table 
Nil  mber 

paragraphing 

comments 

self-evaluation 

<.  1% 

6 

variables 

1st 

self-evaluation 

<5% 

8 

variables 

program 

2nd 

self- evaluation 

<1% 

9 

variables 

3rd 

self-evaluation 

<2.5% 

10 

variables 

1st 

self-evaluation 

<.  5% 

15 

variables 

locality 

program 

1  st 

self-evaluation 

<2.  5% 

variables 

3rd 

self-evaluation 

<5% 

1  7 

variables 

1st 

modification 

<5% 

1  8 

variables 

locality 

2nd 

modification 

<2.5% 

1  9 

control  flow 

1st 

self-evaluation 

<5% 

24 

paragraphing 

1st 

self-evaluation 

<5% 

24 

control  flow 

2nd 

self-evaluation 

<5% 

25 

control  flow 

3n3 

self-evaluation 

<5% 

26 

variables 

1  st 

sel f- evaluation 

<5% 

32 

variables 

2nd 

self-evaluation 

<5% 

33 

locality 

1st 

quiz 

<5% 

37 
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Chapter  5 

CONCLUSION?  AND  DIRECTIONS  FOR  FUTURE  RESEARCH 


The  need  for  programming  techniques  and  language 
designs  that  will  aid  in  the  production  of  large  systems 
which  car.  be  easily  understood  and  modified  is  now  widely 
acknowledged.  We  contend  that  we  must  establish  concrete, 
quantitative  evidence  of  those  factors  that  contribute  to 
program  complexity.  We  are  also  now  convinced  that  such  a 
task  is  not  an  easy  one. 


As  mentioned  in  the  introduction,  our  original  goal 
when  we  began  this  research  was  to  obtain  enough  data  to 
conclusively  determine  the  effects  of  all  of  our  suggested 
program  complexity  factors  on  understanding  and 
modifiability.  It  soon  became  apparent  that  not  only  would 
our  limited  resources  severely  restrict  our  efforts  but  also 
that  no  standard  methodologies  existed  that  could  be 
directly  applied  for  obtaining  such  data.  Consequently,  the 
major  goal  of  our  work  became  the  establishment  of  a 
suitable  methodology. 


In  Chapter  3,  we  proposed  a  methodology  for  future 
experiments.  We  described  the  components  in  detail, 
including  in  some  cases  their  development  throughout  our 
series  of  experiments.  We  also  discussed  the  capabilities 
and  limitations  of  the  methodology.  The  major  limitation  of 
the  methodology  is  the  size  of  program  that  can  be  studied 
using  it.  In  Chapter  4, 


we  discussed  the  results  of  various 
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experiments  that  we  performed.  Based  on  the  evidence 
presented  in  these  chapters,  we  contend  that  our  methodol 
meets  the  requirements  of  feasibility  and  credibility  as 
forth  in  the  introduction.  The  development  of 
methodology  as  a  cohesive  unit  from  existing  components 
the  demonstration  that  it  works,  at  least  with  the  subje 
that  were  available  to  us,  is  the  primary  contribution 
this  thesis.  We  cannot  claim  that  our  methodology 
optimal  since  a  methodology  to  measure  abstract  conce 
such  as  understandability  and  modifiability  of  programs 
never  be  totally  validated. 


The  question  of  validating  a  methodology  is  a  classi 
one  in  the  field  of  experimental  psychology.  Three  types 
validity  exist,  "predictive  validity",  "content  validit 
and  "construct  validity"  [Cronbach  1970],  "Predict 
validity",  which  is  not  directly  relevant  to  our  work,  de 
with  how  well  a  given  test  predicts  a  subject's  performa 
in  a  later  endeavour.  "Content  validity"  is  somew 
relevant  to  our  work.  An  example  of  "content  validity" 
how  well  one  of  our  quizzes  covers  the  material  contained 
the  program  that  it  was  designed  to  test.  Much  m 
important  for  our  work,  however,  is  the  question:  Gi 
that  a  quiz  adequately  covers  the  material  of  the  prog 
that  it  was  designed  to  test,  how  well  does  it  measur 
subject's  "understanding"  of  the  program?  The  problem 
how  well  a  test  measures  an  abstract  concept  such 
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a  problem  that  does  not  have  a  definite  answer.  To  quote 
Cronbach : 

"Sometimes  the  tester  starts  with  a  test  that,  he 
wishes  to  understand  better.  Sometimes  he  starts  with 
a  concept  for  which  he  wishes  a  measuring  instrument. 
The  interpretation  of  a  test  is  built  up  very 
gradually,  and  probably  is  never  complete.  As 
knowledge  develops,  we  arrive  at  a  more  complete 
listing  of  the  influences  that  affect  the  test  score, 
and  at  some  estimate  of  the  strength  of  each  influence. 
At  present,  the  interpretation  of  even  the  best- 
established  psychological  tests  falls  far  short  of  the 
ideal — this,  because  theories  of  ability  and 
personality  are  incomplete  and  hazy. 

"Whereas  predictive  validity  is  examined  in  a 
single  experiment,  construct  validity  is  established 
Through  a  long-continued  interplay  between  observation, 
reasoning,  and  imagination." 

In  light  of  this,  we  cannot  hope  to  completely  validate 
our  methodology.  Nonetheless,  our  methodology  is  both 
feasible  and  credible,  and  as  such  can  be  used  to  study 
psychological  complexity  of  computer  programs.  Further 
research  into  the  methodology  itself  is  also  possible  and 
perhaps  desirable. 
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We  believe  that  a  great  deal  of  investigation  is  needed 
into  the  psychological  complexity  of  computer  programs.  The 
limited  number  of  subjects  available  to  us  has  prevented  us 
from  carrying  out  the  extensive  research  that  we  had 
originally  planned.  Our  first  recommendation  for  future 
research  is  to  repeat  experiments  nine  and  ten,  using  our 
proposed  methodology,  with  several  different  groups  of 
subjects  with  widely- vary ing  backgrounds.  In  fact,  we  feel 
that  it  is  important  to  control  the  effect  of  background,  as 
much  as  possible,  as  an  additional  factor  in  at  least  some 
of  the  experiments  in  order  to  see  if  the  effects  of  the 


other  factors 

are 

sensitive 

to 

the  backgrounds 

of  rhe 

subjects.  As 

we 

mentioned 

in 

Chapter  4,  paragraphing. 

control  flow. 

and 

choice  of  control  constructs  in  a 

prog  ram 

are  all  factors 

that  may 

be 

extremely  sensitive 

to  the 

backgrounds  of 

the 

subj  ects. 

Our  next 

recommendation 

is 

to  experiment  with 

further 

levels  of  the  factors  that  ve  have  already  studied.  In 
particular,  further  investigation  with  more  levels  of 
comments  is  needed  to  determine  more  precisely  what 
constitutes  a  meaningful,  well-placed  comment.  Further 
investigation  with  various  styles  of  paragraphing  is  needed 
and  also  the  interaction  of  comments  with  paragraphing  needs 
to  be  studied  further.  Further  investigation  is  needed  with 
more  levels  of  control  flow  to  determine  if  some 
"structured"  control  constructs  are  better  than  others,  and 
also  (as  mentioned  above) ,  to  determine  the  sensitivity  of 
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the  effect  of  control  flow  to  the  background  of  the 
subjects.  Locality  of  data  references  definitely  needs  to 
be  studied  further  with  different  methods  of  localizing  the 
data  references.  In  particular,  we  recommend  an  experiment 
with  two  levels  in  which  both  levels  have  the  same  number  of 
subroutines,  but  at  one  level  the  subroutines  are  arranged 
by  the  data  structures  affected  and  at  the  other  level  they 
are  arranged  randomly.  However,  as  will  be  discussed  below, 
we  believe  that  this  factor  is  sensitive  to  the  size  of  the 
program.  Finally,  the  choice  of  variable  names  can  be 
studied  further  to  determine  more  precisely  what  constitutes 
a  mnemonic  name  and  what  are  optimal  characteristics  of 
names.  After  these  studies  have  been  completed,  other 
factors  in  our  list  of  complexity  factors  should  be  studied 
in  a  similar  fashion. 

Our  final  recommendation  is  that  after  the  effects  of 
the  various  factors  have  been  determined  for  small  programs, 
similar  studies  should  be  made  for  larger  programs.  Certain 
factors,  such  as  locality  of  data  references,  will  have 
different  effects  on  large  programs  than  on  small  ones. 
Some  factors  may  have  effects  that  are  in  the  same  direction 
on  large  programs  as  on  small  ones,  but  it  is  not  at  all 
clear  whether  the  complexity  produced  by  the  factor  will 
remain  constant,  grow  linearly,  grow  exponentially,  etc., 
with  the  size  of  the  program.  Only  experimentation  will 
determine  this,  although  our  methodology  cannot  be  directly 
applied  to  extremely  large  programs. 
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Throughout  all  these  experiments 
the  procedures  and  measures  used 
However,  our  work,  as  described  in  t 
substantial  basis  for  future  research 


that  we  have  proposed, 
may  undergo  changes, 
his  thesis,  provides  a 
in  this  area. 
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Appendix  A. 


EXPERIMENT  1 
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Appendix  A  -  Experiment  1 


Factors  being  studied: 

Comments  at  two  levels: 

1)  present 

2)  absent 

Variable  Names  at  two  levels: 

1)  mnemonic 

2)  short,  meaningless 

Paragraphing  at  ■‘-wo  levels: 

1)  indentation  to  reflect  program  structure 

2)  no  indentation 

Programs  used:  Quickersort  (Appendix  K-1) 

"Wirth’s"  Eight  Queens  (Appendix  K-2) 

Language  used:  PL/I 

Number  of  subjects:  16  (1  per  cell) 

Source  of  subjects:  first  year  undergraduate  computer 

science  course 

Procedure:  Subjects  were  instructed  to  read  the  program  and 

hand-simulate  as  much  of  its  execution  as  they 
could  in  thirty  minutes,  keeping  track  of  the 
values  of  all  variables.  They  were  then  asked 
for  a  self-evaluation  of  their  understanding  of 
the  program  and  asked  to  fill  in  the  blanks  in  a 
paragraph  describing  the  program.  They  were 
then  given  a  questionnaire  about  the  experiment. 


Significant  results:  None  at  <5%  level 


Appendix  B 
EXPEF IMENT  2 
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Appendix  B  -  Experiment  2 

Factors  being  studied: 

Comments  at  two  levels: 

1)  present 

2)  absent 

Variable  Names  at  two  levels: 

1)  mnemonic 

2)  short,  meaningless 
Paragraphing  at  two  levels: 

1)  indentation  to  reflect  program  structure 

2)  no  indentation 

Programs  used:  Quickersort  (Appendix  K-1) 

Postfix  (Appendix  K-3) 

Language  used:  ALGOL- W 

Number  of  subjects:  48  (3  per  cell) 

Source  of  subjects:  second  year  undergraduate  computer 

science  course 

Procedure:  Subjects  were  instructed  to  read  the  program  and 

hand- simulate  as  much  of  its  execution  as  they 
could  in  thirty  minutes,  keeping  track  of  the 
values  of  all  variables.  They  were  then  asked 
for  a  self-evaluation  of  their  understanding  of 
the  program  and  asked  to  fill  in  the  blanks  in  a 
paragraph  describing  the  program.  They  were 
then  given  a  questionnaire  about  the  experiment. 

Significant  Fesults: 

Comments : 

comments  significantly  increased  the  total  number 
of  values  hand-simulated  and  the  number  of  errors 
made  in  hand-simulation 
Variable  Names: 

mnemonic  variables  significantly  decreased  the 
total  number  of  values  hand- simulated  and  the 
number  of  errors  made  in  hand-simulation 
Paragraphing:  none  at  <5%  level 

Interactions  between  Factors: 

significant  interaction  between  comments  and 
paragraphing  on  the  correct  number  of  values 
hand-simulated  and  on  the  self-evaluation 

significant  interaction  between  comments  and 
variable  names  on  the  number  of  errors  made  in 
h and- Simula tion 

Nots:  This  experiment  is  described  in  detail  in  TWeissman 

1  973  ]. 
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Appendix  E  -  Experiment  2 


Table  1 :  Paw  Dat a 

Columns:  A  -  Hand  Simulation  (Number  of  Correct  Values) 

B  -  Hand  Simulation  (Tonal  Number  of  Values) 

C  -  Hand  Simulation  (Number  of  Errors) 

D  -  Fill  in  the  Blanks  (Scale:  C-10) 

F  -  Self-Evaluation  (Scale:  0-9) 

F  -  Course  Grade 


Version:  C 

1 
2 
3 
n 

5 

6 
7 


No  Comments,  No  Paragraphing,  No  Mnemonics 
No  Comments,  No  Paragraphing,  Mnemonics 
No  Comments,  Paragraphing,  Mnemonics 
No  Comments,  Paragraphing,  Mnemonics 
Comments,  No  Paragraphing,  No  Mnemonics 
Comments,  No  Paragraphing,  Mnemonics 
Comments,  Paragraphing,  No  Mnemonics 
Comments,  Paragraphing,  Mnemonics 


replicate  1 


proqram  -  version 

A 

B 

C 

D 

E 

F 

g-sort-G 

69 

69 

0 

8 

1 

59 

g- sort- 1 

9 

13 

4 

0 

2 

65 

g-sort-2 

37 

4  0 

3 

6 

3 

79 

g- sort- 3 

18 

27 

9 

2 

3 

56 

g-sor t-4 

44 

55 

11 

4 

1 

57 

q- sort-5 

38 

40 

2 

1C 

5 

97 

q-sort-6 

26 

7  0 

44 

9 

2 

9  3 

q-sor t-7 

2  C 

37 

17 

10 

4 

77 

postf i x- 0 

34 

44 

10 

5 

5 

9  1 

postfix- 1 

38 

4  0 

2 

6 

1 

9  2 

postf ix-2 

3  0 

36 

6 

5 

2 

75 

postf ix-3 

38 

4  0 

9 

A* 

5 

7 

77 

postf i x- 4 

1  36 

151 

15 

8 

8 

88 

postf ix- 5 

62 

63 

1 

8 

5 

36 

postfix- 6 

26 

3  1 

5 

2 

2 

6  3 

postf i x- 7 

28 

30 

2 

2 

3 

64 
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Peplicate  2 


program  -  version 

A 

B 

C 

D 

E 

F 

g- sort-0 

14 

44 

30 

0 

3 

75 

q-sort- 1 

20 

39 

19 

4 

2 

79 

q-sor t-2 

36 

43 

7 

2 

6 

82 

q- sort-3 

25 

53 

28 

9 

5 

55 

q-sor t-4 

14 

59 

45 

r 

u 

1 

8  C 

q-sort-5 

28 

47 

19 

3 

2 

59 

q-sor i-6 

1  5 

56 

41 

10 

3 

83 

q-sor t-7 

1  9 

52 

33 

2 

1 

7  1 

postfix- 0 

25 

31 

6 

1 

0 

73 

postfix- 1 

29 

45 

16 

6 

3 

84 

postf  i  x-  2 

34 

41 

7 

8 

1 

69 

postfix-3 

19 

24 

5 

3 

3 

59 

postfix-4 

4  1 

46 

5 

2 

8 

5  8 

postf i x- 5 

43 

43 

o 

7 

6 

77 

postf ix- 6 

25 

30 

5 

8 

8 

9  3 

postfix-7 

22 

24 

2 

3 

6 

74 

Pe£licate_ 

_3 

program  -  version 

A 

B 

C 

D 

E 

F 

q-sor t-0 

28 

58 

30 

10 

7 

87 

q-sort- 1 

14 

1  9 

5 

10 

2 

79 

q-sort-2 

15 

34 

19 

8 

3 

5  1 

q-sor t-3 

19 

20 

1 

6 

3 

69 

q- so r t-4 

18 

53 

35 

10 

5 

84 

q-sort-5 

39 

50 

1 1 

10 

7 

8  3 

q-sort-6 

2  1 

52 

31 

5 

2 

37 

q- sort-7 

25 

32 

7 

8 

2 

79 

post  fix-0 

20 

20 

0 

1 

1 

6  3 

postf ix- 1 

26 

30 

4 

2 

1 

77 

postf ix- 2 

16 

I7 

1 

0 

0 

48 

postfix-  3 

40 

40 

0 

10 

8 

79 

postf ix- 4 

26 

34 

8 

8 

4 

6  3 

postfix- 5 

52 

57 

5 

9 

6.5 

90 

postfix- 6 

29 

49 

20 

7 

6 

68 

postf  i  x-7 

35 

4  1 

6 

3 

3 

66 
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Apper.dix  B  -  Experiment  2 


Table  2:  Effects  on  Hand-Simulation  (Number  of  Correct  Values) 

F-Ratio  Significance 
Level 


(a)  Main  Effects 

Means  for  Factor  Comments 
A-bsent  Present 

27.9  33.9  1,4 

Means  for  Factor  Paragraphing 
Absent  Present 

35 . 2  26.7 

Means  for  Factor  Variables 
Absent  Present 

32.  5  29.4 

Means  for  Factor  Program 
Q-Sort  Postfix 

25.2  36.6 


2.8 


.  4 


5.1  <5% 


(b)  Significant 


Absent 

Comments 

Present 


Interactions 

Paragraphing 
Absent  Present 

26.2  29.7 

44.2  23.7 


5.7  <2.5Jb 


(c)  Within  Cells  Variance:  302.  87 
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Appendix  E  -  Experiment  2 


Table  3:  Effects  on  Fand-Simulat ion  (Number  of 


"P-  Ratio 


(a)  Main  Effects 

Means  for  Factor  Comments 
Absent  Present 

9.0  15.4  5.4 

Means  for  Factor  Paragraphing 
Absent  Present 

11.7  12.  6  .  1 

Means  for  Factor  Variables 
Absent  Present 

16.0  8.3 

Means  for  Factor  Program 
Q-Sort  Postfix 

18.8  5.6 


7 . 9 


23.3 


(b)  Significant  Interactions 

Variables 
.Absent  Present 


Absent 

Comments 

Present 


10.0  8.0 
22.0  8.7 


4.  2 


E  rror  s) 

Sign  if ican 
Level 


<5% 


<U 

<.  H 


<5% 


(c)  Within  Cells  Variance:  89,83 
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A.ppendix  E  -  Experiment  2 


Table  4:  Effects  on  Hand-Simulation  (Total  Number  of  Values) 

F-Ratio  Significance 
Level 


(a)  Main  Effects 


Means 

for  Factor 
A_bsen  t 

Comme  nt  s 
Present 

36.9 

49.  3 

5.  7 

<2,5% 

Means 

for  Factor 
A.bsen  t 

Paragraphing 
Pr  esen  + 

46.9 

39.  3 

2.  1 

Means 

for  Factor 
A.bsen  t 

V  ariables 
Present 

48.5 

37.7 

4.  3 

<5% 

Means 

for  Factor 
Q-Sor t 

Program 

Postfix 

4  4.0 

42.  2 

.  1 

(b)  Within  Cells  Variance: 


323. 31 
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Appendix  B  -  Experiment  2 


Table  5:  Effec+s  on  Fill  in  the  Blanks 


Patio  Significance 
Level 


(a)  Main  Effects 


Means  for  Factor  Comments 
A.bsent  Present 

5.1  6.0  1.0 

Means  for  Factor  Paragraphing 
Absent  Present 

5.2  5.8  .4 

Means  for  Factor  Variables 
Absent  Present 

5.3  5.7  .2 

Means  for  Factor  Program 
Q-Sort  Postfix 

6.1  5 .  C  1.3 

(b)  Within  Cells  Variance:  1C. 28 
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Appendix  P  -  Experiment  2 

Table  6:  Effects  on  Self-Evaluation 


F-Ratio  Significance 
Level 


(a) 

Fain  Effects 

Means 

for  ^actor  Comments 
Absent  Present 

3.2  4.0 

3.8 

<ra 

Means 

for  Factor  Paragraphing 
Absent  Present 

3.4  3.8 

,  7 

Means 

for  Factor  Variables 
Absent  Present 

3.4  3.8 

.  6 

Means 

for  Factor  Program 
Q-Sort  Postfix 

3.1  4.1 

5.2 

<5% 

(b) 

S ign if icant 

Interact  ions 

Paragraphing 

Absent  Present 

Absent 

Comments 

Present 

2.1  4.2 

4.-7  3.4 

00 

V 

m 

r— 

<.  IS 

(c)  Within  Cells  Variance:  2.43 


- 1 


r, 
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Appendix 

C  - 

Experiment  3 

Factor  be 

ing 

stud ied : 

Cont  rol 

Flow  at  four  levels: 

1) 

"structured"  constructs,  " 
paragraphed  listing 

simple"  control  flow. 

2) 

"unstructured"  constructs, 
paragraphed  listing 

"simple"  control  flo 

3) 

"unstructured"  constructs, 
unparagraphed  listing 

"simple"  control  flo 

4) 

"unstructured"  constructs, 
unparagraphed  listing 

"complex"  control  fl 

Wr 


cw 


D rograms  used : 


Heapsort  (Appendix  K-4) 

"Naur's”  Eight  Queens  (Appendix  K-5) 


Language  used:  PL/I 

Number  of  subjects:  16  (2  per  cell) 

Source  of  subjects:  graduate  course  on  software  engineering 

Procedure:  Subjects  were  instructed  to  read  the  program 

until  they  were  prepared  to  be  tested  on  it. 
They  were  then  asked  how  well  they  felt  they 
understood  the  program  and  then  given  a  ten- 
question  quiz  about  the  program.  The  subjects 
were  next  instructed  to  hand- Simula te  the 

program,  keeping  track  of  the  value  of  all 
variables.  They  were  then  asked  how  well  they 
felt  they  understood  the  program  and  then  given 
the  same  quiz  as  before.  Subjects  were  allowed 
to  work  at  their  own  pace,  having  been  told  the 
total  time  allotted  for  the  experiment,  and  were 
asked  to  record  the  time  that  they  began  and 
finished  each  section  of  the  experiment:. 

None  at  <5%  level 


Significant  Fesults: 
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Appendix  D  -  Experiment  h 


Factor  being  studied: 

Locality  of  Data  Deferences  at  two  levels: 

1)  use  of  subroutines 

2)  straight-line  program 


Programs  used: 


"Wirth's"  Eight  Queens  (Appendix  K-2) 
"Dijkstra ' s"  Graph  Plotter  (Appendix  K-6) 


Language  used:  PL/I 

Number  of  subjects:  12  (3  per  cell) 

Source  of  subjects:  fourth  year  undergraduate  engineering 

science  students  in  a  compiler  writing 
course 


Procedure 


The  subjects  were  given  fifteen  minutes  to  read 
the  program,  ten  minutes  for  the  first  self- 
evaluation  and  quiz,  thirty  minutes  to  hand- 
simulate  the  program,  and  ten  minutes  for  the 
second  self-evaluation  and  quiz.  The  subjects 
were  then  given  thirty  minutes  to  make  a 
specified  modification  to  the  program.  They 
were  then  given  ten  minutes  for  a  third  self- 
evaluation  and  quiz. 


Significant  Results:  None  at  <5%  level 
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Append ix 

F  - 

Fxpe  rime 

Fact  or 

be 

ing 

stud ied : 

Me 

t  h 

od  o 

f  study 

i) 

hand-si 

2) 

reading 

3) 

subject 

P  rogram 

s 

used 

:  Great 

Short 

Languag 

e 

used 

:  ALGOL 

Number 

of 

sub 

jects : 

Source  of  subjects: 
Procedure : 


The  subjeers  were  giver  : 
the  programs  and  ten  minu- 
evaluation  and  quiz, 
thirty  minutes  for  a  d< 
program  using  one  of  th: 
then  given  ten  minutes 
evaluation  and  quiz,  tl 
specified  modification  to 
minutes  for  a  third  self-* 


’hey 


oftw 

are 

en 

gi 

nee 

r 

ing 

een 

minu 

1 9 

s 

to 

r 

ead 

for 

the 

firs 

t  s 

lf- 

we 

re 

th 

en 

g 

i 

ven 

led 

study 

of 

the 

methods. 

Th 

ey 

w 

ere 

r  a 

se 

CD 

nd 

s 

If- 

y  minute 

3 

to 

ma 

k 

e  a 

program 

t 

a 

nd 

ten 

uati 

on  a 

nd 

q 

uiz 

• 

Significant  Results :  None  at  <5%  level 
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Appendix  F  -  Experiment  6 


Factor  being  studied: 

Variable  Names  at  two  levels: 

1)  mnemonic 

2)  shortened  mnemonic 

Programs  used:  Heapsort  (Appendix  K-4) 

Precedence  Function  (Appendix  K-9) 

Language  used:  PL/I 

Number  of  subjects:  8  (2  per  cell) 

Source  of  subjects:  fourth  year  undergraduate  engineering 

science  students  in  a  compiler  writing 
course 


Procedure : 


The  subjects  were  given  fifteen  minutes  to  r 
the  program,  ten  minutes  for  the  first  se 
evaluation  and  quiz,  thirty  minutes  to  ha 
simulate  the  program  keeping  track  of 
values,  ten  minutes  for  the  second  se 
evaluation  and  quiz,  fifteen  minutes  to  make 
specified  modification  to  the  program,  and 
minutes  for  the  third  self-evaluation  and  qu 


ead 

lf- 

nc- 

all 

lf- 

a 

ten 

iz. 


Significant  results: 

Variable  Names: 

mnemonic  variables  were  significantly  better  t 
shortened  mnemonic  variables  on  all  three  se 
evaluation  s 

Interaction  between  Factors: 

significant  interaction  between  variable  na 
and  program  on  first  self-evaluation 


han 

lf- 


mes 


Note:  This  experiment  is  described  in  detail  in  [Weissman 

1  97h b  ]. 
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Appendix  F  -  Experiment  6 


Table  7;  Paw  Data 


Columns :  A 

B 
C 
D 
v\ 

F 


First 

Second 

Third 

First 

Second 

Third 


Self -Evaluation  (scale: 

Self-Evaluation  (scale: 
Self-Evaluation  (scale: 
Quiz  (scale:  0-10) 

Quiz  (scale:  C-10) 

Quiz  (scale:  0-10) 


0-9) 

0-9) 

0-9) 


Version:  C  -  Mnemonic  Variables 

1  -  Shortened  Mnemonic  Variables 

Replicate  1 


prog  ram  -  version 

h-sort-0 
h -sort- 1 
prece  dence-0 
precedence- 1 


ABC 

2  5  5 

3  2  2 

9  9  9 

5  3  5 


D  E  F 

6  7  7 

4  3  3 

5  3  8 

8  9  9 


Replicate  2 


program_2_ version 

h-sorr-0 
h-sor t- 1 
precedence-0 
Drecedence-1 


ABC 

3  6  8 

4  4  4 

8  9  9 

a  5  4 


D  E  F 

o  3  i : 

7  3  3 

8  10  13 

7  7  3 
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Appendix  F  -  Experiment  6 

Table  8:  Fffects  or.  First  Self-Evaluation 

F-rati 


(a)  Main  Effects 

Means  for  Factor  Variable  Names 
Mnemonic  Shortened 

5.5  4.0  9.0 

Means  for  Factor  Program 

H-Sort  Precedence 

3.0  6.5  49. C 


(b)  Significant  Interaction 

P  rogr am 

H-Sort  Precedence 
Mnemon ic  2.5  8.5 

V  ariables 


Sho  rtened  3.5  4.5 

(c)  Within  Cells  Variance:  .50 


25. C 


’able  9:  Effects  on  Second  Self-Evaluation 


F- ratic 


(a)  Main  Effects 

Means  for  Factor  Variable  Names 
Mnemonic  Shortened 

2.  25  3.  5  25.0 

Means  for  Factor  Program 

H-Sort  Precedence 

4.  25  6.  5  9.0 

(b)  Within  Cells  Variance:  1.13 


Significance 

Level 


<5% 


<•  5% 


<  1  % 


Significance 
L  evel 


<1% 


<5% 
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A.ppendix  F  -  Experiment  6 


Table  1C:  Effects  on  Third  Self-Evaluation 

E-ratio 


(a)  Main  Effects 

Means  for  Factor  Variable  Names 
Mnemonic  Shortened 

7.75  3.75  18.3 

Means  for  Factor  Program 

H-Sort  Precedence 

4.75  6.75  4.6 

(b)  Within  Cells  Variance:  1.75 


Table  11:  Effects  on  First  Quiz 


F-ratio 


(a)  Main  Effects 

Means  for  Factor  Variable  Names 
Mnemonic  Shortened 

a. ^5  6.5  .9 

Means  for  Factor  Program 

H-Sort  Precedence 

4.25  7.  C  2.2 


Significance 

Level 


<2. 5$ 


<10% 


Significance 

Level 


(b)  Within  Cells  Variance 


6.88 
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A.ppendix  F  -  Experiment  6 


Table  12:  Effects  on  Second  Quiz 


F-ratio  Significance 
Level 


(a)  Main  Effects 


Means  for  factor  Variable  Names 
Mnemonic  Shortened 


8.  25 


8.  0 


.  1 


Means  for  Factor  Program 

H-Sort  Precedence 


7.  75 


8.  5 


1.0 


(b)  within  Cells  Variance:  1.13 


Table  13:  Effects  on  Third  Quiz 


F-ratio  Significance 
Level 


(a)  Main  Effects 

Means  for  Factor  Variable  Names 
Mnemonic  Shortened 

8.V5  8.25  .3 

Means  for  ^actor  Program 

F-Sort  Precedence 

8.25  8.75  .3 


(b)  Within  Cells  Variance: 


1.75 
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Appendix  G  -  Experiment  i 


Fac-or  being  studied: 

Method  of  study  at  three  levels: 

1)  subject's  own  method 

2)  reading  a  trace  of  the  program 

3)  reading  a  trace  and  an  execution  profile  of 
the  program 

Programs  used:  Precedence  Function  (Appendix  K-9) 

Permutation  (Appendix  K-1C) 

Language  used:  ALGOL-F 

Number  of  subjects:  12  (2  per  cell) 

Source  of  subjects:  graduate  course  on  software  engineering 

Procedure:  The  subjects  were  given  fifteen  minutes  to  read 

the  program,  which  they  were  told  might  contain 
errors  and  inefficiencies.  They  were  next  given 
ten  minutes  for  a  self-evaluation  and  quiz,  and 
twenty-five  minutes  for  a  detailed  study  of  the 
program.  They  were  then  given  ten  minutes  for  a 
second  self-evaluation  and  quiz  and  forty-five 
minutes  to  correct  the  errors  and  ineff icienoies 
in  the  proaram. 

Significant  Pesults:  None  at  <5%  level 
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Appendix  H  -  Fxperimei  t  8 

Factor  ^eing  studied: 

Variable  Names  at  three  levels: 

1)  mnemonic 

2)  shortened  mnemonic 

3)  pronounceable,  meaningless 
Locality  of  Data  Deferences  at  two  levels: 

1)  use  of  subroutines 

2)  straight-line  program 

Programs  used:  Quickersort  (Appendix  K-1) 

"Firth* sM  Eight  Queens  (Appendix  K-2) 

Language  used :  PL/I 

Number  of  subjects:  36  (3  per  cell) 

Source  of  subjects:  second  year  undergraduate  computer 

science  course 

Procedure:  The  subjects  were  given  five  minutes  to  real  the 

program.  They  were  next  asked  for  a  self- 
evaluation  and  given  fifteen  minutes  for  a 
detailed  study,  using  any  method  they  wished, 
and  a  quiz.  They  were  then  asked  for  a  second 
self-evaluation,  and  given  twenty  minutes  to 
make  +hree  specific  modifications  to  the 
program.  Finally,  they  were  given  ten  minutes 
for  a  third  self-evaluation  and  a  second  :juiz. 

Significant  Pesults: 

Variable  Names: 

mnemonic  variables  were  significantly  better  than 
shortened  mnemonic  variables  on  the  first 
modification 

mnemonic  variables  were  significantly  better  than 
meaningless  variables  on  the  first  and  third 
self-evaluations 

shortened  mnemonic  variables  were  signif icna tly 
better  than  meaningless  variables  on  the  first 
self -e valuation 
Locality  of  Data  Deferences: 
none  a4:  <5f  level 


ppendix  R  -  Experiment  8 


Interaction  between  Factors: 

significant  three-way  interaction  between 
variable  names,  locality  of  data  references,  and 
program  on  first  self-evaluation 

significant  interaction  between  variable  names 
and  locality  of  data  references  on  second 
modification 
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Appendix  H  -  Experiment  8 


Table  14 


Raw  Tata 


Columns:  A  -  First  Self-Fva luat ion 

F  -  Second  Self-Fvaluat ion 
C  -  Third  Self -pvaluation 
B  -  pirst  Modification 
S  -  Second  Modification 
F  -  Third  Modification 
G  -  First  Quiz  (Scale: 

H  -  Second  Quiz  (Scale: 


(Scale : 
(Scale : 
(Seal  a: 
(Scale  : 
(Scale : 
(Scale : 
0-10) 
0-10) 


0-9) 
C  —  9 ) 


0-5) 

0-5) 

0-5) 


-9) 


Version 


C  - 
1  - 

3  - 
u  - 
e  _ 


Subroutines,  Mnemonic  Variables 
Subroutines,  Shortened  Mnemonics 
Subroutines,  Meaningless  Variables 
No  Subroutines,  Mnemonic  Variables 
No  Subroutines,  Shortened  Mnemonics 
No  Subroutines,  Meaningless  Variables 


R  e£l  icat  e_1_ 


2rog r am _2_ version 


c 


D 


q-sor+ -0 
q- sort- 1 
q-sort-2 
g-sort- 3 
g-sort- 4 
q- sor i-5 
queen  s-C 
queens- 1 
queen  s-2 
queen  s- 3 
queens-4 
queens-5 


1 

c; 

r\ 

\j 

2 

2 

1 

3 

2 

o 

1 

3 

1 


2 

2 

1 

4 

3 
1 
1 

4 
4 

2 

r 

O 


3 

2 

1 

6 

3 
2 
n 

5 

4 
2 

6 
3 


5 

5 

5 

5 

5 

5 

3 

0 

r\ 

3 

0 


0 

1 

4 
1 

2 

3 

1 

r 

V 

1 

5 

4 


5 

3 

4 

5 
5 

c: 


0 

5 


3 
0 
1 

0 

3 

2 ,  5 
1 

5 

2  .  5 

f, 

6 

1 


3.  5 

1  .  5 

5 

2.  5 
z. 

5 

2  „  5 
7,  5 

£  C 

vJ  t  -  > 

3  .  5 


qrcqrs  m_2_ve  r  sior. 


R e^licat; 
B  C 


D 


q-sort-0 
q-sert - 1 
q-sort-2 
q- sort-3 
cr—  sort-4 
g-sort- 5 
queen s-G 
queens- 1 
queens-2 
queen  s- 3 
queen  s-4 
queens- 5 


1 

2 

1 

4 

c 
2 
2 
C 
r , 
r 
1 


1 

C 

1 

4 

3 

r 

3 

2 

1 

2 

Q 

2 


1 

r 

2 

4 


3 

2 

1 

7 


5 

5 

3 

3 

q 

3 

4 
2 

3 

1 

2 


1 

2 


0 

a 

4 
1 
1 

0 

5 


5 

3 

4 
3 

5 

3 

4 


0 


n 

1 

n 

1 

2 

3 
2 
2 
6 
0 

4 


2.5 


1 

1 

2 

1  .  5 

2 
■) 

5  ,  5 

3 

4 

5 

3.  5 

4,  5 
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prog r a m_-_ version 

q-sort-C 
g- sort- 1 
q-sor t-2 
q-sor t-3 
q-  so  r  r  -  4 
q-sort-5 
queens-C 
queens- 1 
queen  s-2 
queen  s- 3 
queens- 4 
queens-5 


Peplicate  3 
A  B  C  D 

2  2  2  3 

3  2  11 

2  5  5  5 

4  3  2  5 

5  3  5  0 

C  0  0  5 

3  5  6  1 

1  3  6.52 

C  C  1  1 

1.  5  2  3  0 

235  0.5 

112  1 


E 


2 

0 

0 

u 

0 

5 

5 

5 

5 

0 

4 

4 


F  G  i 

0  0  3 

1  1.5  0 

3  5  5 

5  0,5  0.5 

3  0  0  .5 

4  C  0 

0  3.5  4  .5 

0  7  4,5 

0  0  3 

0  7  7 

0  2,5  4 

0  2.5  5.5 


Table  15:  Effects  on  ^irst  Self-Evaluation 


F- Patio  Significance 
Level 


(a)  Main  Effects 


Means  for  Factor  Data  Locality 
Subroutines  Straight 


1 . 8 


1.7 


,  03 


Means  for  Factor  Variable  Names 
Mnemonic  Shortened  Meaningless 


2.  1 


2.4 


.  8 


8,  7 


between  levels  1,3:  17.4 
between  levels  2,3:  12.5 

Means  for  Factor  Program 

Q-Sort  Queens 


.  1 


1  .  4 


3  .  1 


(b)  Within  Cells  Variance 


1. 08 


< .  5  % 

<,  U 

< .  5  % 


<10% 
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Table  16:  Effects  or.  Second  Self-Evaluation 

E-Ratio  Significance 
Level 


(a)  Main  Effects 

Means  for  factor  Fata  Locality 
Subroutines  Straight 


2.2  2.3  .05 


Means  for  Factor  Variable  Names 
Mnemonic  Shortened  Meaningless 

2.  6  2.5  1.6  1.8 

between  levels  1,3:  3.1  <10% 

Means  for  Factor  Program 

Q-Sort  Queens 

2.1  2.4  .5 


(b)  Within  Cells  Variance:  2.03 
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Appendix 
Table  17: 

(a)  Main 


-  Experiment  P 

Effects  on  Third  Self-Evaluation 

F-Patio  Significance 
Level 

Effects 

Means  for  Factor  Data  Locality 
Subroutines  Straight 

2,8  3.1  ,3 

Means  for  Factor  Variable  Names 
Mnemonic  Shortened  Meaningless 

3.  6  3.  2  2.0  2.4 

between  levels  1,3:  5.2 

Means  for  Factor  Program 

Q-Sort  Queens 

2.  3  3.5  3.7 


<10£ 

<5% 

<5% 


(b)  within  Cells  Variance: 


3.  44 
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Table  18:  Effects  on  First  Modification 

E-Ratio  Significance 
Level 


(a)  Main  Effects 

Means  for  Factor  Data  Locality 
Subroutines  Straight 

2.8  2.6  .2 

Means  for  factor  Variable  Names 
Mnemonic  Shortened  Meaningless 

3.  3  1.8  2.  9  3.  1 

between  levels  1,2:  4.9 

Means  for  Factor  Program 

0-Sort  Queens 

3.8  1.6  18.1 


<10% 

<5% 


< .  1  % 


(b)  Within  Cells  Variance: 


2.  46 
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Table  18:  Effects  on  Second  Modification 


'p  -  Patio  Significance 
Level 


(a) 


Main 


Means  for  Factor  Data  Locality 
Subroutines  Straight 

2.2  2.0  .  1 


Means  for  Factor  Variable  Names 
Mnemonic  Shortened  Meaningless 

2.2  1.8  2.3  .2 

Means  for  Factor  Program 

Q-Sort  Queens 

1.3  2.9  6.8 


(b)  Significant  Interaction 

V ariable  Names 
1  2  3 


4.  3 


1  3,  3 

Locality 

2  1.  C 


2.  C 

1 . 7 


1 . 3 


3.  3 


<2.5.? 


<2,5? 


(c)  Within  Cells  Variance: 


3.25 
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Tabl~  20:  Effects  or.  Third  Modification 


F-patio  Significance 
L  e  vel 


(a)  Main  Effects 


Means  for  Factor  Data 
S  ubro  ut ines 


Locality 
Stra igh  t 


1,  8 


2.3  1.3 


Means  for  Factor  Variable  Names 
Mnemonic  Shortened  Meaningless 


2.5  1.7  1.9  1.2 


Means  for  Factor  Program 

Q-Sort  Queens 

3.7  .4  52.7 


(b)  Within  Cells  Variance:  1.83 


<.  U 


Table  21*.  Effects  on  First  Quiz 


F-Patio  Significance 
Level 


(a)  Main  Ff^ects 

Means  for  Factor  Pata  Locality 
Subroutines  Straigh4: 

2.2  2.2  0 

Means  for  Factor  Variable  Names 
Mnemonic  Shortened  Meaningless 

2.  1  2.5  1.8  .4 

Means  for  Factor  Program 

Q-Sort  Queens 

1.2  3.1  7.5  <2  5S 

(b)  Within  Cells  Variance:  4.29 
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Table  22:  Fffects  on  Second  Quiz 


F-Tatio  Significance 
Level 

(a)  Main  Fffects 

Means  for  Factor  Fata  Locality 
Subroutines  Straight 

2.9  3.4  .8 

Means  for  Factor  Variable  Names 
Mnemonic  Shortened  Meaningless 

3.6  2.9  3.0  .7 

Means  for  Factor  Program 

Q-Sort  Queens 

1.8  4.6  25.9 


(b)  Within  Cells  Variance:  2.63 


<.  U 
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Factors  being  studied: 

Control  Flow  a4:  three  levels: 

1)  "structured”  constructs,  "simple"  control  flow 

2)  "unstructured"  constructs,  "simple"  control  flow 

3)  "unstructured"  constructs,  "complex"  control  flow 
Paragraphing  at  two  levels: 

1)  indentation  to  reflect  program  structure 

2)  no  indentation 

Programs  used:  Keapsort  (Appendix  K-4) 

Shortest  Paths  in  a  Graph  (Appendix  K-8) 

Language  used:  PL/I 

Number  of  subjects:  24  (2  per  cell) 

Source  of  subjects:  third  year  undergraduate  computer 

science  course  and  two  graduate 
students 


Procedure  : 


The  subjects  were  given  five  minutes  to  real  the 
program.  "hey  were  next  asked  for  a  self- 
evaluation  and  given  fiften  minutes  for  a 
detailed  study,  using  any  method  they  wished, 
and  a  quiz..  They  wer3  then  asked  for  a  second 
self-evaluation,  and  given  twenty  minutes  to 
make  two  specific  modifications  to  the  program. 
Finally,  they  were  given  ten  minutes  for  a  third 
self-evaluation  and  a  second  quiz. 


Significant  Fe suits: 

Control  Flow: 

"structured"  constructs  with  "simple"  control 
flow  were  significantly  better  than 

"unstructured."  constructs  with  "complex"  control 
flow  on  all  three  sel^-e vaiuat ions 
Paragraphing  : 

a  paragraphed  listing  was  significantly  better 
than  an  un oara graphed  listing  on  the  first  self- 
evaluation 

Interaction  between  Factors: 
none  at  level 
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Table  23:  Faw  Data 


Columns:  A  - 

B  - 
C  - 
D  - 

y  _ 


Version:  0  - 

1  - 
2  - 

3  - 

4  - 

5  - 


£rog;ra  m_-_ve  r  sior 


First  Self-Evaluation  (Scale: 
Second  Self-Fvaluation  (Scale: 
Third  Self-Evaluation  (Scale: 
First  Modification  (Scale:  0-5) 
Second  Modification  (Scale:  0-5) 
First  Quiz  (Scale:  C-10) 

Second  Quiz  (Scale:  C-10) 


0-9) 

0-9) 

0-9) 


"structured"  constructs,  "simple"  flaw, 
paragr  aphed 

"structured"  constructs,  "simple"  flow, 
un para  graphed 

"unstructured"  constructs,  "simple"  flow, 
paragraphed 

"unstructured"  constructs,  "simple"  flow, 
unparagraphed 

"unstructured"  constructs,  "complex"  flow, 
paragraphed 

"unstructured"  constructs,  "complex"  flow, 
unpargraphed 

Re£licat  e_  1_ 


A 


C 


D 


h-sort-C 
h -sor t- 1 
h-sor t-2 
h- sort-3 

h-sort-4 

h- sort  -  5 
path- C 
path-  1 
path- 2 
path- 3 
path- 4 
path- 5 


3 

3 

2 

1 

2 

1 

ii 

1 

3 

4 
3 

1 


6 

4 

2 

2 

2 

3 
9 
2 
2 

4 
4 
1 


6 

e 

3 

u 

2 

3 

g 

3 

2 

n 

3 

2 


3 

0 

4.  5 
5 

1 

1 

3 

L 

3 

0 

1 

0 


5 

2.5 

5 

5 

5 

r\ 

5 

2 

3 

2 

3 

2 


8 

2 

4 
3 
2 

5 
7 

3.5 

3 

3.5 

7 

1 


8 

3 
6 
5 

5 

6 
8 

4,  5 

4 

3 , 5 
7 
1 
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prog ram_2_ version  A 

h-sort-0  2 
h-sort-1  3 
h-sort-2  3 
h-sort-3  C 
h-sort-4  1 
h- sort.- 5  2 
path-0  7 
path-1  2 
path-2  2 
path- 3  1 
path-4  1 
path-5  1 


Peplicate  2 

B  C  D  F  F 

2  2  5  5  2 

3  3  0  0  4 

3  5  G  2.5  1 

1  1  3.5  0  2 

1115  5 

1  2  5  5  5 

4  4  0  3  5 

3  4  C  2  0 

2  2  0  1  2 

0  0  0  4  5 

1  3  0  0  1 

C  0  1  3  1 


Table  24:  Effects  on  First  Self-Evaluation 

E - pat io 


(a)  Main  Effects 

Means  for  ractor  Control  Flow 
1  2  3 

3,1  2,0  1.5  46 

between  levels  1,2:  3.5 
between  levels  1,3:  9,9 

Means  for  Factor  Paragraphing 
Present  Absent 

2.9  1.7  5,8 

Means  for  Factor  Program 
w-Sort  Path 

1.9  2.5  1.7 


n 


3 

u 


1.  5 

2 

4 

4 
6 
0 
2 

5 

3,  5 

1 


Significance 

Level 


<5% 

<  1 C  7<j 

<2,5% 


<5% 


(b)  Within  Cells  Variance:  1,21 
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Table  25:  Effects  on  Second  Self-Evaluation 


(a)  Main  Effects 

Means  for  Pactor 

1  2 

4.1  2.0 

b  e  t  w  e  en 
b  etwe  en 

Means  for  Factor 
Present 

3.2 

Means  for  Factor 
H- Sor  t 

2.  5 

(fc)  within  Cells  Variance: 


F- 

Ratio 

Signif i 
Level 

Control 

Flow 

3 

1. 

6 

4.6 

<5% 

levels 

1,2  : 

4.7 

<10 

levels 

1,3: 

6,  S 

<5% 

Paragra  phing 
Absent 

2.0  2.5 

Program 

Path 

2.7  .  1 

3.  17 


i  nee 
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Table  26:  Effects  on  Third 

Self-Evaluati 

on 

(a)  Wain  Effects 

E- 

Ratio 

S ignif ioa  nee 
Level 

Weans  for  Factor 

1  2 

Control  Flow 

3 

4.5  2,6 

2.  0 

4.  C 

<5^ 

between 

levels  1,3: 

7.  7 

<2 .  5% 

Weans  for  Factor 
Present 

Paragra  ph ing 
Absent 

3.5 

2.  6 

1.  5 

Weans  for  Factor 
H-Sort 

3.  1 

Program 

Path 

3.0 

,  01 

(b)  Within  Ceils  Variance:  3,37 
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'able  2-7:  effects  or.  First  Modification 


F- Ratio  Signific 
Level 


(a)  Main  Effects 


Means  for  Facto: 

1  2 


1 .  4 


2. 


Control  Flow 
3 

1.  3 


Means  for  factor  Paragraphing 
Present  Absent 


1.  8 


1  .  3 


Means  for  Factor  Program 


H  -Sor 


!.  4 


Path 


(b)  Significant  Interaction 

Paragraphing 
1  "  2 


1 

Control  Flow  2 

3 


2.3 
1 . 9 

.  8 


2,  1 

1.  8 


C 

•  J 


.  6 


1 


3,  C 


<2, 5Z 


<K% 


nance: 


nee 


(c)  Within  Cells  Va 


2.  6 C 
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Table  28: 

( a )  Main 


T  -  Experiment  9 


Effects  on  Second  Modification 

F-Eatio  Significance 
Level 

Effects 


Means 

1 

for  Factor 

O 

z- 

Control  ^low 

3 

3.  1 

2.  8 

2,  9 

.  OU 

Means 

for  Factor 
Frese  nt 

ParagraDhing 

Absent 

3.5 

2.3 

2.7 

Means 

for  Factor 
H-Son 

Progr  am 

Path 

3.3 

2,5 

1  ,  2 

(b)  Wirhin  Cells  Variance: 


3.  52 
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Table  2Q:  Effects  or.  Pirst  Quiz 


F- Ratio  Significance 
Level 


(a)  Main 


Means 

1 

for  Factor 
2 

Control  Flow 

3 

3.9 

2.  °  ; 

3.' 4  ' 

.  4 

Means 

for  Factor 
Present 

Paragraphing 
Ab  sent 

■■ 

3.9 

2.  9 

1  .  3 

Means 

for  Factor 
H- Sort 

Program 

Path 

3,6 

3.3 

.  1 

(b)  Interaction 

Paragraph ing 

1  2 

1 , 7 

1 

5.5 

2,4 

Cor.T.rol  ^Icw  2 

2.  5 

3.4 

3 

3.8 

3.9 

(c)  Within  Cells  Variance:  4.77 
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Table  30:  Effects  or  record  Quiz 


F-Ratio  Signif icance 
Level 


(a)  Main  Effects 

Means 

for  Factor 

Control  ^low 

1 

2 

3 

4,6 

3.  6 

3.  9 

.  4 

Keans 

for  Factor 
Present 

P  araq ra  phing 
Absent 

4.  3 

3.3 

3 . 5 

Means 

for  Factor 
B-Sort 

F rogr am 

Path 

4.  3 

3.8 

.  3 

(b)  Interaction 

Paragraph  iter 

1  2 

1  .  7 

1 

6  .  3 

2.  ° 

Control  Flow  2 

■3  a 

* 

3.9 

3 

4.Q 

3,  0 

<13? 


(c)  within  Cells  Variance: 


4.  2° 
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Factors  being 
V  ari able 

D 

2) 

3) 

locality 

1) 

2) 


studied : 

Names  at  three  levels: 
mnemonic 

shortened  mnemonic 
pronounceable,  meaningless 
of  Data  Feferer.ces  at  two  levels: 
use  of  subroutines 
straight-line  program 


Programs  used:  Quicker sort  {Appendices  K-1  and  L) 

"Firth's”  Eight  Queens  (Appendices  K-2  and  L) 


Language  used :  PL/I 

Number  of  subjects:  36  (3  per  cell) 

Source  of  subjects:  first  year  undergraduate  computer 

science  course 

Procedure:  The  subjects  were  giver  five  minutes  to  real  the 

proqram.  They  were  next  asked  for  a  self- 
evaluation,  given  fifteen  minutes  for  a  detailed 
study  and  ten  minutes  for  a  second  self- 

evaluation  and  a  guiz.  They  were  then  given  ten 
minutes  to  make  two  specific  modifications  to 
the  program.  Finally,  they  were  given  five 
minutes  for  a  third  self-evaluation  and  a  second 
guiz . 

Significant  Results: 

Variable  Names: 

mnemonic  variables  were  signif ican+ ly  ostter  than 
meaningless  variables  on  the  second  seif- 
evaluat ion 

shortened  mnemonic  variables  were  significantly 
better  than  meaningless  variables  on  the  firs* 
and  second  self-evaluation 
Locality  of  Data  References: 

st raig ht- li n<^  programs  were  significantly  better 
than  programs  with,  subroutines  on  the  first  guiz 
when  considering  only  the  first  and  second  iaveis 
of  variable  names 
Interaction  between  factors: 
none  a  +  <^%  level 
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Table  3 1 : 


paw  Data 


Columns:  A  -  First  Sel^-^valuation 

B  -  Second  Self-Fvaluat ion 
C  -  Third  Self-Evaluation 
D  -  First  Modification  (Scale: 
F  -  Second  Modification  (Scale: 
f  -  First  Quiz  (Scale:  C-1r) 

G  -  Second  Quiz  (Scale:  0-10) 


(Scale:  0-9) 

(Scale:  0-9) 

(Scale:  0-9) 

0-5) 
0-5) 


Version 


0  - 
1  - 
a  _ 

3  - 

4  - 
n  - 


Subroutines,  Mnemonic  Variables 
Subroutines,  Shortened  Mnemonics 
Subroutines,  Meaningless  Variables 
No  Subroutines,  Mnemonic  Variables 
No  Sub routines,  Shorte ned  Mnemonics 
No  Subroutines,  Meaningless  Variables 


F  eplicat  e__1 


pr og r a m_-_ version 


C 


D 


r. 


q- sort-0 
q-sor t-  1 
q-sor t-2 
q-sor t-3 
q- sor  t- 4 
q- sort- 5 
queens-0 
queen  s- 1 
queens- 2 
queans-3 
queen  s-4 
queen  s-5 


1 

5 

3 

c 

c 

2 

3 

2 

1 

0 

3 

1 


5 

4 

5 

6 
2 
4 

3 

1 

6 

3 

2 


2 

4 

4 

r. 

6 

3 

4 
3 

1 

7 

3 

2 


5 

5 

3 

3 

3.  5 

A 

2 

1 

2 


C 

r\ 

0 

4 

2 

4 

A 

\J 

4 

5 

1 

5 


0 


2 

6 

0 

1 

1 

0 

5 


D 

5 

2.5 

2.5 


2.  5 
8 

0  i  5 

1 

2 

0.  5 
6  5 
2.5 
2.  5 


program  -  version 


?epl icat e_2 
B  C 


D 


G 


q-sor t-C 
q-sor t- 1 
q - sor t- 2 
q-sor t- 3 
q-sort -4 
q-sort -5 
queen  s-C 
queens-1 
queer s-2 
queens- 3 
queen  s-^ 
queens-5 


5 

4 

r 

v. 

A 

3 

A 

1 

3 

2 

2 

6 
r 


3 
5 
1 
1 

4 
4 
q 
4 

3,  5 

2 

—7 

1 


4 

A 

3 
1 

4 

3 
9 

4 
3 

2 

r 


5 

5 

n 

5 

4 

5 
4 

A 

1 

0 

2 


1 

n 

0 

0 

1 

3 

0 

0 

0 

3 

1 


3 

1 

1 

1 

0 

5 

0 

3 

3 

4 
2 


5 

5 


A 

V 

3 

1  .  5 
1 
1 


3 . 5 
3 .  5 

4. 5 
2,  5 
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F eplicate_3 


progr am_-_ver sion 

A 

■Q 

C 

D 

E 

F 

G 

q-sor  t-0 

-'i 

5 

4 

0 

0.  5 

0 

1 

q-scrt- 1 

2 

3 

1 

C 

r 

KJ 

n 

0 

q-sor t-2 

2 

2 

3 

5 

3 

0 

C.  5 

q-sor  t. -3 

4 

5 

3 

5 

0 

1 

1 

q-sor t- 4 

1 

2 

1 

0.5 

C 

1 

1 

q-sort-5 

C 

1 

1 

5 

0 

u 

0 

q  ueen  s-O 

c, 

4 

2 

0.5 

2 

2 

2 

queens- 1 

r 

r- 

o 

C 

0 

n 

a 

queens-2 

1 

1 

1 

0.5 

5 

4.5 

4 . 5 

queen. 3- 3 

4 

1 

1 

1 

3 

2 . 5 

2  .  5 

queens- 4 

3 

3 

2 

0.  5 

3 

1 

1  .  5 

queer  s- 5 

2 

2 

1 

r .  5 

3 

2 

2 

Table  32:  Effects  on  First  Self-Evaluation 

F- Ratio  Significance 
Level 


(a)  Fain  Effects 

Means  for  Factor  Fata  Locality 
Subroutines  Straight 

2.4  2.4 

Means  for  ractor  Variable  Names 


Mnemonic  Shorte 

red  Meaningl 

2.8  3.1 

1 . 3 

3 . 5 

<  5% 

fc  et  ween 

levels  1,3: 

4.0 

<10 

between 

levels  2,3: 

8.  8 

<  1  % 

Means  for  Factor 
Q-Sort 

2.  6 

Frogram 

Queens 

2  ° 

,  6 

(b)  Within  Cells  Variance:  2.97 


LO  LO  1/ 
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Table  33:  Effects  on  Second  Self -Fvaluat ion 


F-Ratio  Significance 
Level 


(a)  Main  Effects 


Means  for  Factor  Da  +  a  Locality 
Subroutines  Straight 


3.  4 


3.2 


.  1 


Means  for  Factor  Variable  Names 
Mnemonic  Shortened  Meaningless 


a.  C 


3,8 


2.  0 


3.7 


between  levels  1,3:  6,3 
between  levels  2,3:  6.3 

Means  for  Factor  Program 

Q-Sort  Queens 


3 .  a 


3.  1 


.  2 


(b)  Within  Cells  Variance:  3,67 


<5^ 


<2  . 
<2. 


m  io 
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Table  34:  Effects  on  Third  Self-Evaluation 


F-Ratio  Significance 
Level 


(a)  Main  Effects 

Means  for  Factor  'Data  Locality 
Subroutines  Straight 

3.1  2.8  ,1 

Means  for  Factor  Variable  Names 
Mnemonic  Shortened  Meaningless 

3 . 6  3.1  2.2  1.6 

Means  for  Factor  Program 

Q-Sort  Queens 

3.1  2.8  .1 

(b)  Within  Ceils  Variance:  3.97 


Table  35:  Effects  on  First  Modification 


E-Ratio  Signiricance 
Level 


(a)  Main  Effects 


Means 


for  Factor  Data 
S  ubro  ut  ir.es 


Locality 

Straight 


1.8 


2,6  1.7 


Means  for  ractor  Variable  Names 
Mnemonic  Shortened  Meaningless 

2,5  2.2  1.9 


Means  for  Factor  Program 

Q-Sort  Queens 

3.2  1.2  1 0. C 


<  .  5 


(b)  Within  Cells  Variance: 


3.49 
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Table  36  : 

( a )  Main 


-  Experiment  1 r 


Effects  on  Second  Modification 


F-Ratio  Significanc 
Level 


Effects 

Means  for  Factor  Dana  Locality 
Subroutines  Straight 


1.6 


1.7  .  1 


Means  for  Factor  Variable  Names 
Mnemonic  Shortened  Meaningless 

1.7  1.3  2.0  .6 

Means  for  Factor  Program 

0-Sort  Queens 

1,0  2.3  5=7  <2  5'Z 


(b)  Within  Cells  Variance: 


2.  ?1 
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Table  37:  Effects  on  ^irst  Quiz 


F-Ratio  Significance 
Level 


(a)  Main  Effects 


Means  for  ^actor  Para  Locality 
Subroutines  Straight 

1.3  2.2  3,2 

Means  for  Factor  Variable  Names 
Mnemonic  Shortened  Meaningless 


2.  0 


1.7 


1  .  5 


,  a 


Means  for  Factor  Program 

Q-Sort  Queens 


2.5 


8.9 


<  1: 


<  U 


considering  only  the  first  two  levels  of  variable  names 

Means  for  Factor  Data  Locality 
Subroutines  Straight 

1,C  2.6  5,2  <5% 


(b)  Within  Cells  Variance:  2.35 
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Table  38: 

(a)  Main 


-  Experiment  1 C 


A 


Effects  on  Second  Quiz 


) 


E  -  R  a  t  i  o 


Sign  if icance 
Level  i 


Effects 


Means  for  Factor  Data  Locality 
Subroutines  Straight 

1.5  2.4  1.7 


Keans  for  Factor  Variable  Names 
Mnemonic  Shortened  Meaningless 


2.  a 


2, 


1.5  .6 


Means 


for  Factor 
Q-Sort 


Program 

Queens 


1 .  2 


2.3  5.7 


<^% 


(b)  Within  Cells  Variance: 


3.99 
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ONE  VERSION  0^  EACH  PROGRAM  AND  ITS  QUIZZES  AND  MODIFI GAIIONS 
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Appendix  K-1:  Quickerso~t 

adapted  from  [Scowen  1965] 

SOB  71  :  PROCEDURE  (LIST,  UPPER)  ;  1 

2 

/*  THIS  PROGRAM  IS  INTENDED  TO  SORT  AN  ARRAY  */  3 

U 

DECLARE  LTSrr  (*)  FLOAT  DECIMAL;  5 

DECLARE  UPPER  ^IXED  DECIMAL;  6 

7 

DO  WHILE  (STACK_NOT_EMPTY) ;  8 

IF  SEGMENT_SIZE_GPEATER_THAN_TWO  THEN  9 

DO;  10 

CALL  C H 00 S E _ S P LI T_ V AL U E ;  11 

CALL  REARF  AMGE_AROUND_  SPLIT_ VALUE ;  12 

CALL  S  TACK _ LAP.  GER_PIECE  ;  13 

END ;  1 4 

ELSE  15 

DO;  16 

IF  EXA.CTL  Y_TWO  _ELEMENT  S  THEN  17 

CALL  SORT_SEGMENT;  18 

CALL  UNSTACK_N EXT_SEGMENT;  19 

END;  20 

END;  21 

22 

/*  THE  FOLLOWING  PROCEDURES  MANIPULATE  THE  ARRAY  */  2? 

24 

DECLARE  (SPI,IT_INDEX  , TOP  , BOTTOM,  MIDPOINT)  FIXED  DECIMAL;  25 

DECLARE  LOWER  FIXED  DECIMAL  INITIAL(I);  26 

DECLARE  ( SPLIT_V ALU  E ,T EMP)  FLOAT  DECIMAL;  27 

DECLAPE  FOUND  BIT(1)  ;  28 

DECLARE  TRUE  BIT(1)  INITIAL (' 1 ' B) ;  29 

DECLARE  FALSE  BIT(1)  INITIAL  ( '  0 1  B)  ;  30 

3  1 

CHOO  SE_  SPLIT  _VA.L  UE :  PROCEDURE;  32 

MIDPOI NT=  (UPPEF  +  LOWEF.)  /2  ;  3  3 

SPLIT_^ALUE=LIST (MIDPOINT)  ;  34 

LIST (MIDPOINT)  -LIST (LOWER)  ;  35 

END;  36 
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7  EAR  RAN  G E_AR  OUND_ SPLIT _V ALUE  :  PROCEDURE  ; 

BOTTOM =LOW  ER+ 1 ; 

TOP=UPPER; 

DO  WHILE  (ROTTOM<=TOP)  ; 

FOUND=FALSE ; 

DO  WHILE  (BO  TTOM  <=TC  P  5  -.FOUND)  ; 

IF  LIST (BOTTOM) <=SPLIT_VALUE  THEN 
BOTmOM=ROTTOM+ 1 ; 

ELS^ 

FOUND=TRUE; 

END; 

EOUND=EALSE ; 

DO  WHILE  (BD  TTOM<=TOP  8  -.FOUND)  ; 

IE  LIST  (TOP)  >=SPLIT_VALUE  THEN 
TOP=  mOP- 1 ; 

ELSE 

FOUND =TRUE ; 

END; 

IF  FOUND  THEN 
DO  ; 

TEMP=LI S I (BOTTOM) ; 

LIST  (BOTTOM)  =LX ST  CROP)  ; 

LIST  (TOP)  =TEMP  ; 

TOP=TOP- 1 ; 

END; 

END; 

SPLIT_INDEX=TOF ; 

LIST (LOWEP) =LT  S T ( S PLIT_IN DEX)  ; 

LIST (SPLIT_INDEX) = SPLIT_VALUE; 

END; 

SOET_SEGMENT :  PROCEDURE ; 

IF  LIST (LOWER)  >LIST  (UPPER)  THEN 
DO; 

TEMP=LI ST (LOWER) ; 

LIST  (LOWER) =  LI  ST  (UPPER)  ; 

LIST  (UPPER) =TE  MP ; 

END; 

WND; 

SEGM  ENT_  SIZ  E_GR  EATER  _T  HA.  N_TW  0  :  PROCEDURE  RETURNS  (BIT  ( 1 )  )  ; 
RETURN  ( UPPER- LO WER> 1 ) ; 

END  ; 


EX ACTLY_TWO_ELEMFNTS :  PROCEDURE  RETURNS (BIT ( 1 )) ; 
RETURN  (UPPEF.-LO WEE=  1 )  ; 

FNP ; 


37 

38 

39 
4  0 

4  1 

42 

43 

44 

45 

46 

47 

48 

49 
5C 

5  1 

5  2 

53 

54 

55 

56 

57 

58 

59 

6  C 
61 
62 

63 

64 

65 

66 

67 

68 
69 

7  0 

71 

72 
7  3 

74 

75 

7  6 

77 

78 

79 

8  C 
81 
82 
83 
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Apper.rlix  K-1:  Quickersort 

/*  THE  FOLLOWING  PROCEDURES  MANIPULATE  THP  STACK  Op  SEGMENTS  */  84 

8  5 

DECLARE  NUMFZR_OF_S EGMENTS  PIXED  DECIMAL  INITIAL  (1);  86 

DECLARE  1  STACK  (liUPPEH),  87 

2  LOWE  F._BOPNPS  FIXED  DECIMAL,  88 

2  UPPEP_BOUNDS  FIXED  DECIMAL;  89 

9C 

STACK_LARGER_PTECE:  PROCEDURE ;  91 

IE  2* SP LIT_INDEX>U PPER  +  LO W ER  THEN  92 

DO;  9  3 

LOWEF._BOUNDS  (NUNBEK_OF  JSEGMENTS)  =LOWER;  Q4 

UP PER _ POUNDS (NUMBFR_OF_ SEGMENT  S)  =SPLIT_I NDEX- 1  ;  95 

LC WER=  SPLIT_INDEX  + 1 ;  °6 

FND ;  97 

ELSE  98 

DO;  99 

LO WEP_BOUNDS  (NUM3EE_CE_S EGMENTS)  = SPLIT _I N D EX f 1 ;  100 

UDPEE_ BOUNDS (NUMBER_CF_ SEGMENTS)  =UP?ER;  101 

U  PPER=  SPLIr'_INDEX-1 ;  102 

END;  103 

NUMBER_OE_  SEGMENTS  =  NUMBER_OF_SEGMENT S  + 1 ;  104 

PND;  105 

1  06 

UNSTACK_N EXT_SEGMENT :  PROCEDURE;  107 

NUMBEr_0F_SEGMFNTS=NUM3ER_0F_SEGMENTS- 1 ;  108 

IF  STACK_N OT_F  MPTY  THEN  1C9 

DO;  110 

LO W  FR  =  L OWE F_BOU NDS  ( NU M BE R_OE_ SEGMENTS)  ;  111 

UPPEP=UPPEF_BOUNDS  (NUMBER_OF_S  EGMENTS)  ;  112 

END ;  113 

END;  1  1  4 

115 

STACK_NOT_pM PTY :  PROCEDURE  RETURNS  (BIT ( 1 ))  ;  116 

RETURN  (NUMBEE_OF_SEGMENTS>0) ;  IIP 

END;  113 

1  19 

FND  SORT;  120 
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Appendix  K-1 :  Quickersort 


fill  in  the  blanks  (as  used  in  experiments  one  and  two) 

This  program  sorts  an  array  using  the  following  method 

The  value  at  the _ of  the  array  is  chosen  anl  is 

used  to  _  the  array  by  placing _ values 

_ it  in  the  array  and _ values  _ 

it.  This  is  done  with  a  double  loop  with  one  counter  going 

_  through  the  segment  and  the  other  counter  going 

_ .  The  above  process  is  repeated  on  each  new 

segment  until  a  segment  of  size  _  is  reached  in 

which  case  nothing  is  done,  or  a  segment  of  size  _ 

is  reached  which  is  then  sorted. 


quiz  (as  used  in  experiments  eight  and  ten) 

1,  After  choosing  a  particular  element,  how  is  an  array 
segment  partitioned  using  this  element? 


2  Consequently,  where  is  this  element  placed  in  the  array? 

3.  The  section  of  the  proqram  which  partitions  an  array 
segment  consists  of  an  outer  DO-WHILE  loop  which  contains 
two  inner  DO-WHILF  loops.  What  does  the  first  inner  loop 
do? 


4.  What 

5.  What 

6.  What 

n .  When 
segment? 

8  What 

9.  What 

1C,  What 


does  the  second  inner  loop  do? 
does  the  outer  loop  do? 
is  the  purpose  of  statement  35? 
does  the  program  decide  not  to  furthe 

does  it  do  in  this  case? 

is  the  purpose  of  statement  64  and  why 

is  +he  stack  used  for? 


partition  a 


is  it  valid? 
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2:£§£.ions  (all  three  used  in  experiment  eight,  first  two 
used  in  experiment  ten) 

1.  If  lines  87  -  89  of  the  program  were  replaced  by  the 

f  ollowing : 

DECLARE  STACK  (1: UPPER, 1:2)  FIXED  DECIMAL; 
what  other  changes  would  have  tc  be  made  tc  the  program? 

2.  If  instead  of  directly  sorting  only  segments  of  size  2, 
we  also  want  to  directly  sort  segments  of  size  3,  wnich 
lines  would  have  to  be  changed?  (Indicate  the  line  numbers 
in  the  space  below;  you  need  not  indicate  the  actual 
changes. ) 

3.  Make  modifications  within  lines  32  -  36  of  the  program 
to  use  a  random  element  of  the  segment,  instead  of  the 
middle  one,  for  splitting  the  segment.  (You  may  assume  a 
function  exists  for  generating  random  numbers.) 
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Appendix  K-2:  "birth's"  Eight  Queens 

adapted  from  [Wirth  1971] 

QUEENS:  PPOCEDUE  E  OPTION S  ( MA IN)  ;  1 

2 

/*  THIS  PROGRAM  IS  INTENDED  TO  SOLVE  THE  PROBLEM  OF  PLATING  */  3 

/*  EIGHT  QUEENS  ON  A  CHESSBOARD  SUCH  THAT  NO  TWO  QUEENS  LIE  */  4 

/*  ON  THE  SAME  ROW,  COLUMN,  OR  DIAGONAL  */  5 

6 

DECLARE  (ROW, COLUMN)  FIXED  DECIMAL;  7 

DECLAPE  SAFE  BT T ( 1 )  ;  8 

DECLARE  TRUE  BIT(1)  INITIAL  ( '  1 ’ B)  ;  9 

DECLARE  FALSE  BIT(1)  INITIAL  ( ' 9 ' B)  ;  1C 

1  1 

CALL  CO NS IDF  R_EI F ST_COLUMN ;  12 

DO  WHILE  (iI,AST_COLUMN_DONE  S  -»REGRESS_OUT_OF_FIRST_COLrJ  MN)  ;  13 
S AFE=E AL  SE ;  14 

CALL  TRY_COLUMN;  15 

IE  SAFE  THEN  16 

DO;  17 

CALL  SET_QUFEN;  18 

CALL  CONSIDEP_NEXT_COLUMN;  19 

END;  2C 

ELSE  21 

CALL  REGRESS  ;  22 

END;  23 

IF  LAST_COLUMN_DONE  THEN  24 

CALL  PRINT_ SOLUTION;  25 

26 

TRY_COL  UMN :  PROCEDURE;  27 

DO  WHILE  (-iSA.FF  &  ->LA.S  T_S QUAPE)  ;  28 

CALL  ADVANCE_POINTER ;  29 

CALL  TEET_SQUAEE;  3C 

END;  31 

END;  32 

33 

REGRESS:  PROCEDURE;  34 

CALL  PECON SIDED _PRIOR_COLU  MN ;  35 

IE  -.REG RES  S_OUT_OE_FIR ST_COLUMN  THEN  36 

CALL  REM OVE_ QUEEN;  37 

END;  38 

39 
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/*  THE  FOLLOWING  PROCEDURES  HANDLE  THE  BOAF.D  REPRESENTATION  */  40 

4  1 

DECLAPE  POSITION (8)  FIXED  DECIMAL ;  42 

4  3 

CONSIDE?_FIFST_COLUMN:  PPOCFDUF.E;  44 

COLUMN  =  1 ;  4  5 

ROW  =  0 ;  46 

END;  47 

48 

CONS IDER_NEXT_COLUMN :  PF  OCEDUFE ;  49 

POSITION (COLUMN) =ROW;  5C 

COLUMN =COL UMN+  1  ;  51 

ROW=0;  52 

END;  53 

5  4 

PECO  N  SIDEF_PRIOP._COLU  MN  :  PROCEDURE;  5F 

COLUMN =COL UMN -  1  ;  56 

IF  COLU MN>  = 1  THEN  57 

P.OW=POSImION  (COLUMN)  ;  58 

END;  59 

60 

ADV ANCE_POI NTER :  PROCEDURE;  61 

ROW=ROW+ 1 ;  62 

END;  63 

6  4 

PR  I  Nirr_  SOLUTION :  PROCEDURE;  65 

PUT  SKIP  LIST  (POSITION  (1)  , POSITION (2)  , POSITION (3)  ,  66 

POSITION (4)  , POSITION  (5)  , POSITION (6)  , POSITION  (7)  ,  6  7 

POSITION (8) )  ;  68 

END;  69 

70 

LAS  T_S  QUARE  :  PFOCEDUPF  RETUFNS  (BIT  ( 1)  )  ;  71 

RETURN  (F.OF=8)  ;  7  2 

END;  73 

7  4 

LAST_COLUMN_DONF :  PFCCEDUP^  RETURNS  (BIT(1));  75 

RETURN  (COLUMN>8) ;  76 

END;  77 

78 

RFGP FSS_OUT_CF_FIRST_COLUMN :  PROCEDURE  RETURNS  (31 T ( 1 ) *  ; 

RETURN  (CO  LUMN< 1 )  ;  80 

END ;  81 
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/*  T'HE  FOLLOWING  PBOCEDUFES  HANDLE  THE  SAFETY  REPRESENTATION  */  82 

83 

DFCLAFF  POW_SAFE (8)  BIT(1)  84 

INITIAL  (  (8)  ( 1)  •  1  «  B)  ;  8  5 

DECLA'PF  DIAGONAL _ON,E_SA.FE  (2:  16)  BIT(1)  86 

INITIAL  ((15)  (1)  M'B);  87 

DECLARE F  DIAGONAL_TWO_SAFE  (-7:  7)  BIT(1)  88 

INITIAL  ( ( 15)  (1)  « 1 'B) ;  p9 

9  0 

TES  T_SQUARF :  PROCEDURE;  91 

S AFE=R OW_S AFE (FOW)  &  DIAGONAL_ONE_S AFE (ROW+ COLUMN)  92 

f-  DIAGONAL_TWO_SAFE  (ROW-COLUMN)  ;  93 

END;  94 

95 

SET_QUEEN;  PPOCEDUPE;  96 

ROW_SAFS(F.OW)  =  FALSE;  97 

DIA.GONAL_ONE_S AFE  ( FOW  +  COLUMN)  =FALSS;  98 

DI AGON AL_TWO_S AFE (ROW- COLUMN) =FALSF ;  99 

END;  IOC 

1  01 

RFKO  VE_QUFFN  :  PF.  OCEPTJRF  ;  102 

ROW_SA.FE  (F.CW)  =  rrF.UF  ;  1  C  3 

DIAGONAL_CNE_S  AFE ( ROW+  COLUMN) =TRUE ;  1 04 

DIAGONA.L_T  WO_S  AF E  ( F.OW -COLUMN)  =TRUF  ;  1  05 

FND;  1C  6 

107 
1  0  8 


END  QUEENS; 
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fj.ll  j_n  the  blanks  (as  used  in  experiment  one) 

This  program  solves  the  problem  of  placing  _ 

items  cn  a  board  such  that  no  two  of  them  lie  in  the  smae 

_ ,  _ ,  or  _ using  the  follawrng 

method.  Each  of  the  _  is  searched  for  a  safe 

position.  If  such  a  position  cannot  be  found,  then 

to  the  previous _ and  look  for  another 

safe  position. 


quiz  (as  used  in  experiment  four) 

1,  In  what  order  are  the  columns  considered? 

2,  For  any  given  column,  in  what  order  are  the  rows 
con  si der ed? 


3  What  determines  if  a  square  is  safe? 


4,  if  a  state  is  reached  in  which  a  queen  cannot  be  placed 
in  a  given  column,  what  does  the  program  do? 


5. 

W  h  y 

does  the 

index  of  DIAGONAL 1 

_SAFE 

run 

8  . 

Why 

does  the 

index  of  DIAGONAL2 

_SAFE 

run 

7 

f  or 

The 

the 

value 

program 

of  which,  variable  determi 
to  terminate? 

nes 

from  2  to  15? 
from  -7  to  7? 

when  it  is  time 


8,  What  two  conditions  with  regard  to  this  variable  will 
cause  the  program  to  terminate  and  what  does  each  condition 
mean? 

9,  What  must  be  done  when  a  queen  is  placed  on  the  board? 

10,  When  is  a  queen  removed  from  the  board? 


» 
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.hppendix  K-2:  "Wirth's"  Fight  Queens 
quiz  (as  used  in  experiments  eight  and  ten) 

1  Explain  the  data  structure  used  to  represent  the  board, 

2,  What  determines  if  a  square  is  safe? 

3,  Fxplain  the  data  structure  used  no  represent  the  safety 
representation. 

4,  If  a  state  is  reached  in  which  a  queen  cannot  be  placed 
in  a  given  column,  what  does  the  program  do? 

5  What  relationship  holds  between  the  row  and  column  of 

any  two  positions  which  lie  on  the  same  upward  diagonal? 

6,  What  relationship  holds  between  the  row  and  column  of 

any  two  positions  which  lie  on  the  same  downward  diagonal? 

The  value  of  which  variable  determines  when  it  is  time 
for  the  program  +o  terminate? 

8.  What  two  conditions  with  regard  tc  this  variable  will 
cause  the  program  to  terminate  and  what  does  each  condition 
mean? 

9.  What  must  be  done  when  a  queen  is  placed  on  the  board? 
1C.  When  is  a  queen  removed  from  the  board? 


2241122211211  (dS  used  in  experiment  four) 

In  the  space  below  indicate  how  you  would  modify  the 
program  to  find  all  solutions  to  the  eioht  queens  problem 
instead  of  iust  the  one  solution  which  it  currently  finis. 


Appendix  K-2: 


"Firth's"  Fight  Queens 


(all  three  used  in  experiment  eight,  first  two 
used  in  experiment  ten) 

1 .  If  line  42  of  the  program  were  changed  to  the  following: 

DECLA.EE  POSITION  (8,8)  BIT(1); 
wha*  other  changes  would  have  to  be  made  to  the  program? 

2.  If  instead  of  solving  the  eight  queens  problem,  we 
wanted  to  solve  the  six  queens  problem,  which  lines  would 
have  to  be  changed?  (Indicate  the  line  numbers  in  the  space 
below;  you  need  not  indicate  the  actual  changes,) 

3.  Make  modifications  within  lines  12  -  25  of  the  program 
in  order  to  find  all  solutions  to  the  eight  queens  problem. 
Indicate  your  modifications  in  the  space  below, 
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PROCFDUPE  PROGRAM  (STRING(8C)  INSTRING;  STRING (80)  3UTSTRING) 
BEGIN 

STRING (1)  ARRAY  OPSTACK ( 1 ;  : 4 0)  ; 

INTEGER  ARRAY  PRIOSTACK  (1;;U0); 

INTEGER  STACK INDEX  , ININDEX , OUTINDEX ; 

LOGICAL  ENDOFSTRING; 

STRING  (1)  OP; 

INTEGER.  PRIO; 

STRING (B)  OPERATORS; 

INTEGER  ARRAY  PRIORITIES (0 :: 7)  ; 

INTEGER  LOOP; 

LOGICAL  VAFIAELE; 

STRING (26)  ALPHA; 


PEOCEDUFF  ^OP; 

COMMENT 

THIS  PROCEPURF  IS  USED  TO  POP  THE  OPERATOR  STACK  ONTO 
THE  OUTPUT  STRING; 

BEGIN 

I?  STACKINDEX> 1  THEN 
BFGIN 

CUT STRING  (OUTINDEX |  1 )  : =OPSTACK (STACKINDEX- 1 )  ; 
OUTINDEX : =OUTIHDEX+ 1 ; 

STACKINDEX : = ST ACKI NDEX - 1  ; 

END 

ELSE 

ERROR; 

END; 

PROCEDURE  PUSH; 

COMMENT 

THIS  PROCEDURE  IS  USED  TO  PUSH  AN  OPERATOR  ONTO 
THE  OPERATOR  STACK  AND  ITS  PRIORITY  ONTO  THE 
PRIORITY  S T ACK  ; 

BEGIN 

OPSTACK (STACKINDEX)  :=OP; 

PRIOSTACK?! STACKINDEX)  :  =PRIO; 

STACKINDEX : = STACKIND EX + 1 ; 

END; 


PROCEDURE  ERROR; 


COMMENT 

THIS  PROCEDURE  HANDLES  SOME  ERROR  CONDITIONS; 
BEGIN 

WRITE  ("ERROR")  ; 
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GOTO  RETUF.N ; 
END  ; 

COMMENT 

INITIALIZE; 


STAC KIND EX ; = 1 ; 

ININDEX : =C ; 

OUTINDEX : =C ; 

ENDOFSTEING :=FALSE; 

ALPHA: ="ABCDEFGH IJKLKNOPQR  STUVWXYZ" ; 

OPFPATOF.S  :="  ;  ()  =  +-*/"; 

FOR  LOOP  ;  =0  UNTIL  4  no 
PRIORITIES (LOOP) :=LOOP; 

PRIORITIES (5) ; =  4 ; 

PRIOFITIES (6) :=5 ; 

PRIORITIES  (7)  ;  =5  ; 

WHILE  -.ENDOFSTEING  DO 
BEGIN 

COMM  ENT 

SEE  IF  NFXT  CHA.F  ACmER  IS  AN  OPEFATOR  OR  VARIABLE; 

OP: = INST RING  (ININDFX | 1)  ; 

ININDEX: =ININDEX+ 1 ; 

VARIABLE ; = FALSE; 

F OF  LOOP : =°  UNTIL  25  DO 
IF  ALPHA (LOOP | 1) =OP  THEN 
VARIABLE :=TFUE; 

IF  VARIABLE  THEN 
BEGIN 


COMMENT 

PUT  VARIABLE  IN  OUTPUT  STRING; 

CUTSTFING  (OUTINDEX |  1)  :=OP; 
OUTINDEX :=OUTINDEX+ 1 ; 

END 

ELSE 

EEGIN 


COMMENT 

FIND  THE  PRIORITY  OF  THE  OPERATOR; 
LOOP  :  =0  ; 

WHIL7  OP-i=OPERATORS  (LOOP  |  1)  DO 
IF  LOOP<7  THEN 
LOOP  :=LOOP+ 1 
FLSF 

ERROR ; 
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PRIO:=PRTORITTES  (LOOP)  ; 
IF  STACKINDEX=1  THEN 


COMMENT 

STACK  IS  EMPTY ,  PUSH  OPERATOR  ON  IT; 

PUSH 

ELSE 

IE  ?PIO>2  THEN 
BEGIN 


COMMENT 

FOR  THE  OPERATORS  POP  TUE 

STACK  OF  ALL  OPERATORS  WITH  HIGHER  OR  EQUAL 
PRIORITY  AND  PUT  THIS  OPERATOR  ON  THE 
STACK; 

WHILE  (STACK INDEX>1) 

AND  (PRIO<=P  RIOSTACK  (STA.CKINDEX-  1)  )  DO 

POP  ; 

PUSH; 

END 

ELSE 

CASE  PRIO+1  OF 
BEGIN 
BEGIN 


COMMENT 

SEMI-COLON  MEANS  END  0^  STRING  , 

POP  THE  ENTIRE  STACK; 

WHILE  STACKINDEXM  DO 
POP; 

OUTSTRING  (CUTINDEX |  1)  ;=0P; 

ENDOFS  TP.ING  :  =  TRUE ; 

END  ; 

BFGIN 

COMMENT 

LEET  PARENTHESIS  STARTS  SUB -EXPRESSION, 
PUSH  IT  ON  STACK; 

PUSH; 

END; 

BEGIN 


COMMENT 

P I GHT  PARENTHESIS  ENDS  SUB-EXPRESSION, 
POP  STACK  UNTIL  LEFT  PARENTHESIS; 

WHILF  PE  10 STACK  ( STACKINDEX-  1)  -»=  1  DO 
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PND 

END 


END; 


POP; 

STACKINDEX : = STACK INDEX -1 
END; 

END; 


RETUP N; 


» 
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fill  in  the  flanks  (as  used  in  experiment  two) 

This  program  transforms  arithmetic  assignment 
statements  into  postfix  notation  using  the  following  method 
Each  character  is  checked  for  being  a  variable  or  operator 

A  variable  is  immediately  _ .  Operators  are  pished 

on  a  _  such  that  the _ form  a  strictly 

_  sequence.  If  the  sequence  is  broken,  operato 

of  _  and  _ _ are  popped.  Le 

parenthesis  is  qutomat ically  pushed  to  start  a  _ 

E igh  t  paren  thesis  causes  popping  until  the  _ 

is  reached.  Semi-colon  pops  everynhing,  Only  a 
minimal  amount  of  _  checking  is  done. 


n 
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Appendix  K-4:  Heapsori 

adapted  from  [  Knuth  1  ^ "7 3  ] 

SOFT:  PROCEDURE  (ARRAY ,N) ;  1 

2 

/~  THIS  PROGRAM  IS  INTENDED  TO  SORT  AN  ARRAY.  THE  ARRAY  IS  */  3 

/*  TREATED  AS  IF  IT  REPRESENTS  A  BINARY  TREE.  THE  TWO  SONS  */  4 

/*  OF  NODE  ARRAY (I)  ARE  ARRAY (21)  AND  ARRAY (21+1).  */  5 

6 

DECLARE  N  FIXED  DECIMAL;  7 

DECLARE  ARRAY  (*)  ^LOAT  DECIMAL;  8 

DECLARE  (PHASE1 ,  PHAS E2, KEY, SON, FATHER)  FIXED  DECIMAL;  9 

DECLARE  ORDERED  31^(1);  10 

DECLARE  TRUE  BIT  ( 1 )  INITIAL  ( »  1 ' B)  ;  11 

DECLARE  FALSE  BIT(1)  INITIAL  ( '  0 ' 3)  ;  12 

1  3 

PHA.SE  1  =  N/2+ 1  ;  14 

PHAS  E2=N ;  15 

DO  WHILE  (PHASE2> 1) ;  16 

IE  PHAS El > 1  THEN  17 

DC ;  18 

PHA.SE1  =  PHASE  1  -  1  ;  19 

KEY=ARRAY (PHASE1) ;  20 

E ND ;  21 

■c’l  S  E  2  2 

DO;  23 

KEY=AEFAY (PHASE2) ;  24 

AR  RAY (PHASE 2) = APR AY (1)  ;  2  5 

PEAS F2  =  PHASE 2-  1;  2  6 

END;  27 

FATHE?=PPASE1 ;  28 

S  0  N=  2*  FAT HER;  2  9 

OR  DERED=FALSE ;  3C 

DO  WHILE  (SON<=PHASE2  &  -.ORDERED);  31 

IE  SON<PEASE2  TEEN  32 

IE  APPAY  (SON)  < ARRAY  (SON+  1)  THEN  33 

SON  = SON  +  1  ;  3  4 

IE  KEY>=ARP.AY (SON)  THEN  35 

OPDEP.ED=TPUF;  3  6 

"LS^  37 

DO;  3-8 

ARRAY (FATHER) =A?RAY (SON)  ;  3D 

EATEEp=S0N;  -  lx  0 

S0N=2*FATH3R  ;  4  1 

END;  42 

END;  43 

ARRAY (FATHER) =KEY;  44 

END;  45 

END  SORT;  1x6 
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g uiz  (as  used  in 

experiments  three,  six,  and 

nine) 

1,  During 
PHASE  1 ? 

the 

first 

phase , 

what  happens 

no  the 

value 

of 

2.  During 
PHASE  2? 

the 

first 

phase , 

what  happens 

to  the 

value 

of 

3,  During 
PHASE  1? 

the 

second 

ph  as  e. 

what  happens 

to  the 

value 

of 

4.  During 

the 

second 

phas  e. 

what  happens 

to  the 

value 

of 

PHA.SE  2? 

5.  After  the  first  phase  is  completed,  what  property  ioes 
the  tree  have? 

6.  When  is  a  father  node  switched  with  one  of  his  sons? 

7  Which  son  is  he  switched  with? 

8,  During  the  second  phase,  what  happens  to  the  tree? 

9.  Which  element  of  the  array  only  has  one  son? 

1C.  Which  elements  of  the  array  have  no  sons? 

modi f ica tion  (as  used  in  experiment  six) 

In  the  space  below  indicate  how  you  would  modify  the 
program  to  print  out  a  value  of  the  array  when  that  value  is 
assigned  to  its  final  position  in  the  array. 
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mod if ications  (as  used  in  experiment  nine) 

1.  Suppose  line  8  were  changed  to 

DECLARE  ARRAY  (*)  CHAR  (IOC) ; 

^or  efficiency  purposes  we  don't  want  to  move  around  these 
large  character  strings  while  sorting  the  array  so  we 
introduce  another  array  of  indices  info  the  first  array  with 
the  following  declaration: 

DECLARE  PTR_ARPAY  (N)  FIXED  DECIMAL; 

and  we  initialize  it  as  follows: 

DO  1=1  TO  N; 

PTR_ARRAY (I) =1 ; 

END  ; 


What  other  changes  would  you  have  to  make  to  the  program? 

2  If  you  were  to  change  the  program  to  sort  the  array  in 
reverse  order,  which  lines  would  you  have  zo  change 
(Indicate  the  line  numbers  in  the  space  below;  you  need  not 
indicate  the  actual  changes-.  ) 
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adapted  from  [Naur  1972] 

QUEENS:  PROCEDURE  OPTIONS  (MAIN)  ; 

/*  THIS  PROGRAM  IS  INTENDED  TO  SOLVE  THE  PROBLEM  0^  PLACIN 
/*  EIGHT  QUEENS  ON  A.  CHESSBOARD  SUCH  THAT  NO  TWO  QUEENS  LI 
/*  ON  THE  SAME  ROW ,  COLUMN,  OR  DIAGONAL 

DECLARE  (C4 , C5 , M , ROW ,COL , LOW POW , HIGHROW , K)  WIXEU  3INAPY 
DECLARE  ROWVAL (8)  ^IXFD  BINARY; 

DECLARE  EREECOL ( 8)  BIT(1); 

DECLARE  PLACED  BIT  (  1 )  ; 

DECLARE  TRUE  BIT ( 1)  INITIAL  (*1'B); 

DECLARE  FALSE  BIT(1)  INITIAL  ( ' 0 ' B)  ; 

DO  C0L=1  TO  P ; 

EREECOL (COL) =T?  UE ; 

END  ; 

DO  C4= 1  TO  3; 

ROWVAI  (4)  =C4; 

EREECOL (Ca) =EAI SE; 

DO  C5  =  C4+  2  TO  9-C^; 

ROWVAL (5) =C5 ; 

EREECOL (C5)  = FALSE; 

M=1  ; 

LCWR0W=4 ; 

HIGHROW=  5 ; 

ROW =3 ; 

COL=C  ; 

DO  WHILE  (M> C )  ; 

PLACED = FALSE ; 

DO  WHILE  ( COL< 8  &  --PLACED)  ; 

COL=COL+ 1 ; 

IF  EREECOL (COL)  THEN 
DO; 

PLIcED=TRUE; 

DO  K  =  LOW  ROW  TO  HIGHROW; 

IF  ABS  (ROW -K) =ABS (COL-ROWVAL  (K)  |  THEN 
PLACED=FALSE ; 

END; 

END; 

END; 

IF  PLACED  THEN 
DO; 

ROWVAL  (ROW) =COL; 

FP.EECOI.  (COL)  =FALSE; 

END; 


*/ 


C_D  W 


M  W 
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IF  (M=  6  Z  PLACED)  THEN 
DO; 

PUT  SKIP  EDIT  (FO  Vi  VA-L  ( 1 )  ,  ROWVAL  (2)  , ROWVAL (3)  , 

SOW VAL (4)  , ROWVAL  (5)  , ROWVAL (6)  , ROWVAL (7)  , 
ROWVAL  (?)  )  (F  (1)  ,X(5)  ,F  (1)  ,X  (5)  ,F  (1)  ,X  (5)  , 

F  ( 1)  ,X  (5)  ,F(1)  ,X  (5)  ,F(1)  ,X(5)  ,F(1)  ,  X  (5)  ,  *  (  1)  ) 

PL ACED= FALSE; 

END; 

ELSE 

IF  -.PLACED  THEN 
M=N- 1 ; 

ELSF 

M=K+1 ; 

M>3  THEN 
DO; 

LOW  ROW  =  1  ; 

HIPHROW  =  N+  1  ; 

BOW=M+  2 ; 

END; 

ELSE 

DO; 

LC  WEOW=  5-M  ; 

HIGHFO  VT=5  ; 

RCW=4-N ; 

END; 

IF  M>C  THEN 

IF  -.PLACED  'THEN 
DO  ; 

COL=POWVAL  (ROW)  ; 

FREECOL  (COL)  =TRUE ; 

PUT  SKIP  LIST  (’STOP  HAN D- SI  MU L ATI ON  1 )  ; 

END; 

ELSE 
DO  ; 

COL=C ; 

ROWVAL (ROW) =C; 

END; 

END; 

FREECOL  (CP>)  =  T R U E  ; 

END; 

FREECOL (C4)  =TPU  E ; 

ND; 

ND  QUEENS; 
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quiz  (as  used  in  experiment  three) 

1.  Which  two  rows  of  the  chessboard  are  treated  specially? 

2.  This  is  done  in  an  attempt  to  eliminate  some 
solutions. 

3.  What  is  the  relationship  between  M  and  FOW? 

4,  What  does  the  statement: 

IF  ABS  (POW-K)  -APS (COL-ROWVAL (K) ) 
check  for? 

5.  If  a  state  is  reached  in  which  a  queen  cannot  be  placed 
in  a  given  row,  what  do^s  the  program  do? 

6,  After  a  solution  has  been  found  and  printed,  how  does 
the  program  attempt  to  find  the  next  solution? 

7,  What  is  xhe  variable  C4  used  for? 

8,  What  is  the  variable  C5  used  for? 

9,  In  what  order  are  the  rows  considered? 

10.  For  any  given  row,  in  what  order  are  the  col am ns 
consi dered? 
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Appendix  K-6  :  "D  i  jkst ra  '  s'*  Graph  Plotter 


adapted  from  [Dijkstra  1972] 
GRAPH:  PROCEDURE  OPTIONS  (MAIN); 


/*  THIS  PROGRAM  IS  INTENDED  TO  DF AW  THE  GRAPH  OF  THE  FUNCTION  */ 
/*  Y  =  MOD  ( X  ,  2  C  )  */ 

DECLARE  (I,JfK,X,Y)  FIXED  DECIMAL; 

CALL  BUILD; 

CALL  PRINT; 

BUILD:  PROCEDURE; 

CALL  CLEAR; 

CALL  SFT_KAFKS; 

END  BUILD; 

S  E!_ MARKS :  PROCEDURE; 

1  =  0  ; 

DO  WHILE  (K4C  )  ; 

CALL  ADD_KAPF; 

1  =  1+1  ; 

END; 

END  5FT_M AFK  S ; 

ADD_MARK:  PROCEDURE; 

X=FX (I) ; 

Y  =  ^Y (I) ; 

CALL  L INE_M AFK ; 

END  ADD_MAFK ; 

IMAGE_AS_L INES  :  PROCEDURE ; 

PRINT:  ENTRY; 

J=  1 9 ; 

DO  WHILE  ( 7 >  =  0 )  ; 

CALL  LINE_?RINT; 

J=J- 1 ; 

END  ; 

RETURN ; 

CLEAR:  ENTRY; 

J=  1 9 ; 

DO  WHILE  (J>=0) ; 

CALL  LINE_CLEA_E; 

J  =  J-  1  ; 

END; 

RETURN ; 

END  IMAGE_AS_LINES; 
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LINE_AS_CHARACTEPS :  PROCEDUF  E; 

DECLARE  1  IMAGE  STATIC, 

2  LINE  (C  :  19)  , 

3  SYM  (0:39)  CHARACTER  (1) 

LIN  E_P  PINT :  ENTRY; 

K  =  0  ; 

DO  WHILE  (K<4C)  ; 

PUT  EDIT  (LINE  (J)  ,  SYM  (K)  )  (A  ( 1 )  )  ; 

K  =  K+  1  ; 

END; 

PUT  SKIP; 

RETURN  ; 

LIN  E_CLFAP :  ENTRY; 

K  =  C ; 

DO  WHILE  (K<tiD)  ; 

LINE  (J)  . SYM (K) =»  '  ; 

K=K+  1  ; 

END; 

RETURN ; 

LIN  E_MAR  K :  ENTRY; 

LINE  (Y)  .  S v M  (X)  =  *  *  '  ; 

RETURN; 

END  L IN E_AS_CHAR ACTED S  ; 

EX:  PROCEDURE (L)  RETURNS  (EIXED  DECIMAL); 

DECLARE  L  EIXED  DECIMAL; 

RETURN  (L)  ; 

END  EX; 

t?Y:  PROCEDURE  (M)  RFTURNS  (EIXED  DECIMAL); 

DECLARE  M  FIXED  DECIMAL; 

RETURN (MOD (M,2C)  )  ; 

END  FY; 


END  GRAPH; 
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s-  guiz  (as  used  in  experiment  four) 

1.  How  many  lin^s  are  there  in  the  graph? 

2.  How  many  characters  are  there  per  line? 

3.  How  many  points  are  plotted? 

4.  what  is  done  to  the  graph  before  any  points  are  plotted? 
5  What  happens  each  time  a  point  is  calculated? 

6.  Why  are  ail  the  points  calculated  before  any  of  them  are 
print  ed? 

7.  After  all  the  points  have  been  calculated  how  is  the 
graph  printed? 

8.  When  is  the  statement:  PUT  SKIP  executed  and  why? 

9.  It  the  function  being  plotted  were  changed  to  another 
function,  what  run-tiire  error  is  most  likely  to  occur? 

10.  Assuming  that  I/O  is  costly  compared  to  CPU  time,  what 
major  inefficiency  exists  in  the  program? 

(as  US'S'?  in  experiment  four) 

Tn  the  space  below  indicate  how  you  would  modify  the 
program  so  that  the  trailing  blanks  on  each  line  will  not  be 
printed  out. 


-1  75- 


Appendix  K-7:  Greatest  Common  Divisor 


adapted  from  [Bradley  197C] 

PROCEDURF  DIVISOF  (INTEGER  ARRAY  A  (*)  ; INTEGER  VALUE  N 
INTEGER  RESULT  GCD) ; 

BEGIN 

INTEGER  M, TEMP, QUOTIENT; 

INTEGER  ARRAY  MULT  1 , MULT 2  ( 1 :  :N)  ; 

INTEGER  ARRAY  C ( 1 : : 2 , 1 : : 2) ; 

LOGICAL  SIGN; 

COMMENT  THIS  PROGRAM  COMPUTES  THE  GREATEST  COMMON 
DIVISOR  °F  N  NUMBERS; 

M :  =  1 ; 

WHILE  ( M<=  N)  AND  (A(M)=0)  DO 
BEGIN 

MULTI (M) :=C ; 

MULT2 (M) ;=C ; 

M; =  M+ 1  ; 

END; 

P  M>N  THEN 
GCD: =0  ; 

else 

IF  M  =N  THEN 
BEGIN 

GCD: =ABS (A (N) ) ; 

MULm  1  (N)  :  =  C  ; 

MULT 2  (N)  :  =TR  UNCATE  (A  (N) /GCD)  ; 

END 

ELSE 

BEGIN 

SIGN  :  =FAI,SE; 

IF  A  (M)  < C  THEN' 

SIGN :=TRUE ; 

C  (1, 1)  : = ABS ( A ( M ) )  ; 

GCD: =C  (1  ,  1)  ; 

^OR  I ;  =  M  + 1  STEP  1  "NEIL  N  DO 

IF  (C(1,1)->=1)  AND  (A  (I)  ->  =  0)  THEN 
BEGIN 

C  ( 1 , 2)  ;  =1 ; 

C(2, 2)  :=0  ; 

C(2, 1)  :=ABS  (A  (I)  )  ; 

WHILE  C  (2,  1)  -*  =  C  DO 

p  F  o  i }} 

IF  C  (2, 1)  <C  (1  ,  1)  THEN 
BEGIN 

TEMP  :  =C  (2 ,  1)  ; 

c  (2,  1)  :=C(1,  1)  ; 

C (1 ,  1) :=mEMP; 

TEMP  :  =C  (2,2)  ; 
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C  (2,2)  :  =C  ( 1 , 2)  ; 

C  (1,2)  : =  TE  M? ; 

END ; 

QUOTIENT : = TRUNCATE (C (2 , 1 ) /C  ( 1  ,  1 )  )  ; 

C  (2,  1)  :  =C  (2,1)  -QU0TIENT*C(1  , 1)  ; 

C  ( 2 , 2)  :=C  (2,  2)  -  QUOTIENTS  ( 1 , 2)  ; 

END; 

MULT  1  (I)  :=C  (1 , 2)  ; 

MULT  2  (I)  :  =  TP.UNCATE  (  (C  ( 1  ,  1 )  -C  (1  ,  2 )  *GCD|  /A(  I)  ) 
GCD  :  =  C  (1,1); 

END 

ELSE 

EEGIN 

MULTI  (I)  :  =  1 ; 

MUI.T2  (I)  :  =  0 ; 

END; 

IE  SIGN  THEN 

MULT2 ( M)  :  =  -MULT2  (M)  ; 

END 

END  DIVISOE; 
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Appen  d ix  K-7 : 


t  Common 


Divisor 


guiz  (as  used  in  experiment  five) 

1.  The  solution  to  what  simpler  problem  is  use!  to  salve 
the  problem  of  finding  the  greatest  common  divisor  of  N 
numbers? 

2,  After  execution  of  the  first  WHILE  loop,  what  does  M 
conta in? 


3.  What  range  of  values  can  the  greatest  common  divisor 
h  avt? 


4,  What  are  elements  C  (1,1)  and  C(2r1)  set  to  prior  to 
execution  of  the  inner  WHILE  loop  (  i.e.,  WHILE  C(2,1)-*=C)? 


5  After  termination  of  this  loop  (i.e.,  when  C(2,1)=C) 
what  does  C(1,1)  con  a  tin? 


6.  What  does  C(1,2)  contain  at  that  point? 

When  QUOTIENT  is  set,  what  do  we  know  about  the  relation 
between  the  numerator  and  denominator? 


8.  What  do  the  arrays  MULTI  and  MULT 2  contain? 

9-  what  is  the  variable  SIGN  used  for  and  why? 

1C,  Why  is  the  TRUNCATE  function  used  at  several  points  in 
the  program? 


m od±_f ication  (as  used  in  experiment  five) 

In  the  space  below  indicate  how  you  would  modify  the 
program  to  find  the  multipliers,  X(i),  of  the  numbers,  \(i), 
whose  greatest  common  divisor  has  been  found.  The 
multipliers  satisfy  the  property: 

X  (1)  *A(1)  +.  .  .  +X  (i)  *A  (i)  +  .  .  .  X  (N)  *A(N)  =  GCD 
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Appsndix  K-8:  Shortest  Paths  in  a  Graph 

adapted  from  [Johnson  1973] 

GRAPH:  PROCEDURE  (WEIGHT, N, PATH , DISTANCE) ;  1 

2 

/*  THIS  PROGRAM  COMPUTES  T  HF  MINIMUM  DISTANCE  PATH  FROM  */  3 

/*  THE  SOURCE  NODE  (NODE  1)  TO  ALL  THE  OTHER  NODES  OF  A  */  4 

/*  DIRECTED  GRAPH.  THE  GRAPH  IS  PASSFD  ^O  THE  PROGRAM  */  5 

/*  AS  A  WEIGHTED  ADJACENCY  MATRIX.  IF  AN  EDGE  DOES  NOT  */  6 

/*  EXIST  FROM  NODE  I  TO  NODE  J,  THE  ENTRY  IN  THE  MATRIX  */  7 

/*  IS  INFINITY,  */  8 

o 

DECLARE  WEIGHT  (*,*)  FLOAT  DECIMAL;  10 

DECLARE  N  FIXED  DECIMAL;  11 

DFCLAR.F  PATH  (*)  FIXED  DECIMAL;  12 

DECLARE  DISTANCE  (*)  FLOAT  DECIMAL;  13 

DECLARE  NODF_SFT  (1:N*(N-1))  FIXED  DECIMAL;  14 

DECLARE  (NOPE_SET_IN DEX,  CUR PENT_N OD E)  FIXED  DECIMAL;  15 

DECLARE  INFINITY  FLOAT  DECIMAL  INITIAL  (32003);  16 

17 

DO  1=2  TO  N;  18 

DISTANCE (I) =INFINITY;  19 

END;  20 

DISTANCE  (1) =0;  21 

NODS_SET_INDEX=  1  ;  22 

NODE__SET  (NODE_SFT_INDEX)  =1  ;  23 

DO  WHILE  (NODF_SET_INDEX-=C) ;  24 

C  U  E  E  E  N  T_N O  D  E=  N  O  DE_S  ET ( 1 )  ;  25 

DO  1=2  TO  NCDE_SFT_INDEX;  26 

NODF_SET  (1-1)  =NODE_SET (I)  ;  27 

END;  28 

NODE_SET_INDFX=NODE_SET_INDEX- 1 ;  29 

DO  1=1  mO  K;  30 

IF  WEIGHT  (CU?PENT_NCDF,  I)  -»  =  INFINITY  THEN  31 

IF  DISTANCE (I)  >  DISTANCE (CURSENT_NODE)  32 

+  WEIGHT  (CUREENT_NODE, I)  THEN  33 

DO;  34 

DISTANCE (I) =DI STANCE (CURRENT_NODE)  35 

+  WEIGHT  (CURRENT_NODE,I)  ;  36 

PATH (I) =CURRENT_NODE;  37 

NODE_SIT_INDEX=NODE_SET_INDEX+1 ;  38 

NO  DE_S  FT (NODE_SET_INDEX)  =1 ;  39 

END;  40 

END;  ''  4  1 

END;  42 

43 

END  GRAPH;  44 
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Appendix  K-8:  Shortest  Paths  in  a  Graph 
guiz  (as  used  in  experiment  five) 

1.  What  does  the  ith  element  of  DISTANCE  contain? 

2.  When  is  an  element  of  DISTANCE  updated? 

3.  What  does  the  ith  element  of  PATH  contain? 

4.  How  are  the  paths  from  the  source  node  to  the  other 
nodes  stored  in  the  array  PATH? 

5.  What  kind  of  data  structure  does  the  array  NQDE_SFT 
i mple  men  t? 

6.  To  what  is  NODF_SFT  initialized? 

7.  Why  dees  NODF_SET  have  an  upperbound  of  N*(N-1)? 

8  How  is  a  node  selected  from  FODE_SET? 

9,  What  is  done  to  that  node? 

10.  When  is  a  node  added  to  NODE_SFT? 

gudz  (as  used  in  experiment  nine) 

1.  r-7hat  does  the  ith  element  of  DISTANCE  contain  at  any 

point  in  the  program'5 6 7 8 9 

2.  When  is  an  element  of  DISTA.NCE  updated? 

3.  What  does  the  ith  element  of  DISTANCE  contain  when  the 
program  terminates? 

4.  What  does  the  ith  element  of  PATH  contain  when  the 

program  terminates? 

5.  What  kind  of  data  structure  does  the  array  N0DE_ SET 

i mplem  ent ? 

6.  To  what  is  NCDF_SFT  initialized? 

7.  Why  does  NOPF_SET  have  an  upperbound  of  N *  ( N - 1 )  ? 

8.  How  is  a  node  selected  from  N0DE_SET? 

9.  what  is  done  to  that  node? 

1C.  When  is  a  node  added  to  NODE  SET? 
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Appendix  K-8 :  Shortest  Paths  in  a  Graph 


^carions  (as  used  in  experiment  five) 

In  the  space  below  indicate  how  you  would  make  the 
following  modifications  to  the  program: 

1.  Make  the  program  print  out  the  path  found  from 
the  source  node  to  each  of  the  other  nodes, 

2.  Change  the  program  so  that  instead  of  receiving 
an  adjacency  matrix  as  input,  it  instead  receives  a 
list  of  edges  in  the  graph.  The  representation  of 
this  list  is  up  to  you. 


m od i_f  ications  (as  used  in  experiment  nine) 

1.  Instead  of  representing  the  graph  as  an  adjacency 
matrix,  suppose  a  list  of  edges  is  used,  i, e. ,  line  10  is 
replaced  by 

DECLAP E  1  EDGES  (*), 

2  STAPTI NG_NODE  FIXED  DECIMAL, 

2  ENPING_NODE  FIXED  DECIMAL, 

2  WEIGHT  FLOAT  DECIMAL; 

and  an  additional  parameter  M,  the  number  of  edges,  is 
passed  in.  what  changes  would  you  have  to  make  to  the 
program?  (You  may  assume  the  list  of  edges  is  sorted  by 
START ING_NODE,  and  edges  with  the  same  S T A RTING_NODE  are 
sorted  by  ENDI NG_NODE . ) 

2.  Which  lines  would  you  have  to  modify  to  have  the  source 
node  passed  in  as  a  parameter  rather  than  assuming  it  to  be 
node  1?  (Indicate  the  line  numbers  in  the  space  below;  you 
need  not  indicate  the  actual  changes.) 
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Appendix  K-9:  Precedence  Function 


PRECEDENCE:  PROCEDURE  (MATRIX # LEFT, RIGHT, N) ; 

/*  TEIS  PROGRAM  IS  INTENDED  TO  CONVERT  A  PRECEDENCE  MAIRIX  FOR*/ 
/*  A  GRAMMAR  INTO  TWO  PRECEDENCE  FUNCTIONS,  LEFT  AND  RIGHT.  */ 
/*  THE  CODE  IN  THE  (I, J)  TH  ENTFY  OF  MATRIX  WILL  3E :  */ 

/*  PRECEDENCE  OF  SYMBOLS  I,J  CODE  */ 

/*  3  */ 

/*  <  2  */ 

/*  >  1  */ 

/*  NO  RELATE  ON  C  */ 

/*  */ 

/*  THE  FUNCTIONS  WILL  HAVE  THE  PROPERTY:  */ 

/*  PRECEDENCE  CF  SYMBOLS  I,J  */ 

/*  =  LEF'1'  (I)  =RIGHT  (J)  */ 

/*  <  LEFT  (I)  <RIGHT  ( J)  */ 

/*  >  LEFT  (I)  >RIGHT(  J)  */ 

DECLARE  (ftlPIX  (*,*),  LEFT  (*),  RIGHT  (*)  ,N)  FIXED  BINARY; 
DECLARE  CHANGED  BIT  ( 1)  ; 

DECLARE  (I,J)  FIXED  BINARY; 

DECLARE  TRUE  BT T ( 1 )  INITIAL  ( 1  1 ' B)  ; 

DECLARE  FALSE  BIT  (  1)  INITIAL  ( ' 0 ' B)  ; 

DO  1=1  TO  N; 

LEFT  (I)  =1  ; 

RIGHT  (I)  =1 ; 

END; 

CHANG E D=T P UE  ; 

DO  WHILE  (CHANGED) ; 

CHANGED=EALSF ; 

DO  1=1  TO  N; 

DO  J=1  TO  N; 

IF  MATRIX  (I  ,  J)  =  1  THEN 
DO  ; 

IF  LEFT  (I)  <=RIGHT  (J)  THEN 
DO  ; 

LEFT  (I)  =PIGHT  (J)  +1  ; 

CHAN  GED=TRUE ; 

END  ; 

END; 

ELSE 

IF  MATPIX (I, J)  =2  THEN 
DO; 

IF  LEFT  (I)  >  =  RIGHT  (J)  THEN 
DO; 

FIGHT  (J) =L EFT (I)  +1 ; 

CHANGE D=T RUE ; 

END  ; 

FND ; 

ELSE 

IF  MATRIX  (I,  J)  =3  THEN 


1 

2 

3 

4 

5 

6 

7 

8 

9 

1C 

1  1 

12 

13 

14 

1  5 

16 

17 

18 

19 

2C 

21 

22 

23 

24 

25 

26 

27 

28 

29 

3C 

31 

3  2 

33 

34 

35 

36 

37 

38 

39 

4  0 

4  1 

42 

43 

44 

4  5 

46 

47 

48 

49 

50 

51 
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Appendix  K-9:  Precedence  Function 

DO;  52 

IF  LEFT  (I) -.=  PIGHT  ( J)  THEN  53 

DC  ;  54 

IF  LEFT  (I)  <RIGHT  (J)  THEN  55 

LEFT  (I)  =P.IGHT  ( J)  ;  56 

ELSE  57 

FIGHT  (J)  =LEFT  (I)  ;  58 

CHANGED=TRUE;  59 

END;  6C 

END;  61 

FND ;  6 

END;  6 

END;  64 

END  PRECEDENCE;  65 


C\l  O') 


-183- 


Appendix  K-9:  Precedence  Function 


quiz  (as  used  in  experiments  six  and  seven) 

1 Why  is  it  desirable  to  convert  a  precedence  matrix  into 
precedence  functions? 

L 

2  Are  the  precedence  functions  unique? 

3-  In  what  two  cases  is  an  element  of  LEFT  incremented  and 
why? 

u.  In  what  two  cases  is  an  element  of  RIGHT  incremented  and 
why? 

5.  If  MATRIX (I , J) =C  for  a  given  I,J  what  does  rhe  program 
d  o? 

6-  What  causes  the  program  to  terminate? 

Give  an  example  of  a  2x2  matrix  for  which  the  program 
will  go  into  an  infinite  loop. 

8,  Why  does  this  happen? 


222^2=22^:1.211  (as  used  in  experiment  six) 

In  the  space  below  indicate  how  you  would  modify  the 
program  so  that  it  will  not  go  into  an  infinite  loop  when 
given  a  matrix  for  which  precedence  functions  do  not  exist. 


err2rs  (as  seeded  in  experiment  seven) 

1,  line  32  was  changed  to: 

TO  J=I  TC  N; 

2.  line  59  was  deleted 
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Appendix  K-1C:  Permutation 

adapted  from  [Robinson  1967  ] 

BEGIN  1 

2 

COMMENT  THIS  PROGRAM  IS  INTENDED  TO  PRINT  OUT  ALL  3 

PERMUTATIONS  OF  THE  INTEGERS  FROM  1  TO  NUMBER;  4 

5 

INTEGER  NUMBER;  6 

READ  (NUMBER) ;  7 

BEGIN  8 

INTEGER  ARRAY  V ( 1 :: NUMBER) ;  9 

10 

PROCEDURE  PERMUTE  (INTEGER  VALUE  N;  INTEGER  VALUE  K;  11 
INTEGER  ARRAY  VECTOR (*)) ;  12 

13 

COMMENT  THIS  PROCEDURE  PRODUCES  IN  VEITOR  THE  KT H  14 
PERMUTATION  OV  THE  INTEGERS  FROM  1  TO  N;  15 

16 

BEGIN  17 

INTEGER  QUOTIENT, REMAINDER , Z EEC_CO  UN  TEE ,  18 

Cl D_ QUOTIENT , J ;  19 

LOGICAL  FOUND;  20 

FOR  I : = 1  UNTIL  N  DO  21 

VECTOP  (I)  :=C ;  22 

OL  D_QUO  TIENT: =K;  2  3 

FOR  T  :  =N  STEP  -1  UNTIL  1  DO  24 

BEGIN  25 

QUOTIENT :  =  TRUNCATE ( OLD_ QUO  TIEN T/I)  ;  26 

REMAINDER : =01 D_QUO TIENT- QUOTIENT*! ;  27 

ZEP.O_COUNTER:  =  0;  28 

J: =N+ 1 ;  29 

FOUND: =FALSE;  30 

WHILE  -.FOUND  DO  31 

BEGIN  32 

J:=J-i;  33 

IF  VECTOR  (J)  =0  THEN  34 

BEGIN  35 

IE  Z ER 0_ CO U NT ER= REMAINDER  THEN  36 

FOUND: =T RUE  37 

ELSE  38 

Z E R C_C OUN T ER  :  =ZERO_COUNTE?.+  1  ;  39 

END;  43 

EN  D ;  ''41 

VECTOR  (J)  :=I;  4  2 

OLD_QUOTIE NT := QUOTIENT ;  43 

END;  -  44 

END;  45 

46 


-185- 


ppendix  K-10:  Permutation 

INTSGFR  PROCEDUFF  FACTORIAL  (INTSGFR  VALUE  N)  ;  47 

IF  N  =  C  THEN  1  ELSE  N *EACTORT AL (N- 1 ) ;  48 

49 

FOE  COUNT : =0  UNTIL  FACTORIAL (NUMBER) -1  DO  5C 

BEGIN  51 

PEP MUTE  (NUMBER, COUNT, V)  ;  52 

WRITS (V ( 1) ) ;  53 

FOR  LOOP :  =  2  UNTIL  NUMBER  DO  54 

WRITEON  (V  (LOOP)  )  ;  55 

END;  56 

END;  57 

END.  58 
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Appendix  K  —  1 C :  Permutation 

2.uiz  (as  used  in  experiment  seven) 

1.  For  any  given  permutation,  in  what  order  are  the  digits 
placed  in  the  vector? 

2,  Fow  is  the  position  of  a  given  digit  determined? 


3 ■  Which 

statement  in 

the  program 

guarantees  that  two 

digits  won 

'  t 

be  placed  in 

the  same  posi 

tion7 

^ .  In  each 

success ive 

permutation , 

what  happens  to  the 

digit  N? 

5.  What  is  the  function  of  QUOTIENT? 

6,  Why  is  OLD_QUOTIENT  needed? 

7,  What  is  the  function  of  REMAINDER? 

8.  What  is  the  function  of  J? 

Q.  What  is  the  function  of  ZERO_CCUNTER? 

10.  When  QUOTIENT  reaches  zero,  what  happens  to  the 
remaining  digits? 

errors  (as  seeded  in  experiment  seven) 

1,  line  42  was  changed  to: 

VECTOR ( J) :  =  J ; 

2.  lines  47-48  were  deleted  and  line  5C  was  changed  to: 

FOR  COUNT : =C  UNTIL  NUMBER- 1  PO 
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Appendix  I 

ALL  VERSIONS  OF  ^HE  PROGRAMS  USED  IN  EXPERIMENT  10 
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Quickersort  (Subroutines,  Mnemonic  Variables) 

SORT:  PROCEDURE  (LIST, UPPER) ; 

/*  THIS  PROGRAM  IS  INTENDED  TO  SORT  AN  ARRAY  */ 

DECLARE  LIST (*)  FLOAT  DECIMAL; 

DECLARE  UPPER  FIXED  DECIMAL; 

DO  WHILE  (STA.CK_N OT_EMPTY)  ; 

IF  SEGMENT_SIZE_GREATER_THAN_TWO  THEN 
.  DO; 

CALL  CHOOS  F_SPLIT_ VAL  U  E ; 

CALL  R EARRANGE_AROUND_SPLIT_ VALUE; 

CALL  S TACK _L  AR GES_PIEC E ; 

END; 

ELSE 

TO; 

IF  EX ACTLY_T WO_ELFMFNTS  THEN 
CALL  SCRT_SEGMENT ; 

CALL  U NST AC K_N EX T_ SEGMENT; 

END; 

END; 

/*  THE  FOLLOWING  PROCEDURES  MANIPULATE  THE  ARRAY  * / 

DECLARE  (SPLIT_INDSX  , TOP , BOTTOM, MIDPOINT)  FIXED  DECIMAL; 
DECLARE  LOWER  FIXED  DECIMAL  INITIAL (1) ; 

DECLARE  (SPI  UPVALUE,  TEMP)  FLOAT  DECIMAL; 

DECLARE  FOUND  BIT(1)  ; 

DECLARE  TRUP  BIT(1)  INITIAL  (  M ' 3)  ; 

DECLARE  -FALSE  BI  T  ( 1 )  IN ImIAL  ( '  C  ’  3)  ; 

CHOO  SE_ SPLIT _ VALUE;  PROCEDURE; 

MTDPOINT=  (UPPEP  +  LONER) /2; 

SPLIT_VALUF=LIST (MIDPOINT) ; 

LIST  (MIDPOINT)  =LIST (LOWER)  ; 

END; 
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PEAR RAN GE_A.P OUNP_ SPLIT_ VALUE  :  PROCEDURE; 

BOTTOM=LOFER+ 1 ; 

TOP=UPPER ; 

DO  WHILE  (BOTTOM<=TOP)  ; 

EOTJND=FALSE; 

DO  WHILE  (BOTTON<=TOP  Z  -.’POUND)  ; 

IF  LIFT (BOTTOM) <= SPLTT_VALUE  THEN 
BOTT OM=BOTTO N+  1  ; 

ELSE 

^OUN  D=TC  TJ E ; 

END  ; 

FOUND=EALSE; 

DO  WHILE  (BOTTOM<=TOP  &  -nFOUND)  ; 

IF  LIST  (TOP)  >= SPLIT_VALUE  THEN 
TOP=TOP- 1 ; 

ELSE 

FOUND=TRUE; 

END  ; 

IF  FOUND  THEN 
DO; 

TEUP=LIST (BOTTOM) ; 

LIS1-1  (BOTTON)  =L I S T  (TOP)  ; 

LIST  (TOP) =TF  MP ; 
mOP=TOP- 1 ; 

END; 

END  ; 

SPLIT_INDEX=TOF ; 

LIST  (LOWER) =LIST (SPL IT_INDSX)  ; 

LT ST  (SPLIT_INDEX)  =  SPLIT_ VALUE; 

END; 

SORT_SEGMENT :  PROCEDURE; 

IF  LIST (LOWER)  >LIST  (UPPER)  THEN 
DO; 

TEMP=LIST (LOWEF) ; 

LIST (LOWER)  =LIST  (UPPER)  ; 

LIS'71  (UPPER)  =TENP; 

END; 

END; 

SEGM  ENT_SIZV_GREATER_THAN_TW 0 :  PROCEDURE  RETURNS  (BIT  (1)  ) 
RETURN  (UPPER -LOWEPM)  ; 

END; 

EXACTLY_TWO_ELEMFNTS :  PROCEDURE  RETURN S  (BIT  ( 1 ))  ; 

RETURN  (UPPER- LOWE 1= 1 ) ; 

END; 
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/*  I  HE  FOLLOWING  PROCEDURES  MANIPULATE  THE  STACK  OF  SEGMENTS  */ 

DECLARE  NUMBEP_OF_SEGMENTS  FIXED  DECIMAL  INITIAL (1); 

DECLAEE  1  STACK  (1:UPPEP), 

2  LOWE F._BOUNDS  ^IXED  DECIMAL, 

2  UPPEF_BOUNDS  FIXED  DECIMAL; 

STAC  K_LARGEF_PIECE :  PROCEDURE; 

IF  2*SPLIT_INDEX>UPPER+LOWER  THEN 
DO; 

LOWEF_FOUNDS (NUMBER_OF_SEGMENTS)  =LOWER; 

U?PER_EOUNDS (N UMBEFJD  F_S EGMENTS) =SPLIT_I NDEX-  1  ; 
LOWER=SPLIT_INDEX+ 1 ; 

END; 

ELSE 

DO; 

LCWEF_ BOUNDS (NUM3EF_OF_ SEGMENTS) =SPLIT_INDEX+ 1 ; 
UPPE?_EOU NDS  (NUMBEF_OF_S EGMENTS)  =UPPER; 

UPPEF= SPLI T_IN DEX- 1 ; 

END  ; 

NUMEEE_OF_ SEGMENTS =  NU MBEE_OF_ SEGMENTS  +  1 ; 

END; 

UN5TACK_NEXT_SEGMENT:  PROCEDURE; 

NUMBEF  _OF_ SEG M F NT S =NU M BE F_OF_ SEGMENTS- 1 ; 

IF  STACK_NOT_EMPTY  THEN 
DO; 

LOWEF.  =  LONE  P._BOU  NDS  (NUMBER_OF_ SEGMENTS)  ; 

UPPEF  =  UPPER _BOU NDS  (NUMBEF_OF_ SEGMENTS)  ; 

END; 


STACK_NOT_EMPTY:  PROCEDURE  RETURNS  (3IT ( 1 ))  ; 
RETURN  ( NUMBEF_OF_ SEG M ENTS >0) ; 

END  ; 


END  SORT; 
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i c k ^ ^ s o r t  ( Subr  out  ir  es ,  Shortened  Mnemonic  Variables) 
SPT :  PROCEDURE  (LST,UPPR); 

/*  THIS  PROGRAM  IS  INTENDED  TO  SORT  AN  ARRAY  */ 

DECLARE  LSI ( *)  FLOAT  DECIMAL; 

DECLARE  UPPP.  FIX  FD  DECIMAL; 

DO  WHILE  (  STCK_N  0  T_EMPT Y)  ; 

IF  SGMNT_SZ_GPTP_THN_TW  THEN 
DO; 

CALL  CHS_SPLT_VL; 

CALL  P PENG E_AP. ND_S PLT_VL  ; 

CALL  STCK_LRGR_PC; 

END; 

ELSE 

DO; 

IF  SXC TLY_TW_ELMNTS  THEN 
CALL  SPT_SGMNT ; 

CALL  UN STCK_NXT_SGMNT ; 

END  ; 

END; 

/*  THE  FOLLOWING  PFOCEDUEES  MANIPULATE  THE  ARRAY  * / 

DECLARE  (SPI,T_INDX,  TP,  BTTM, MDPNT)  FIXED  DECIMAL; 
DECLARE  LWR  FIXED  DECIMAL  INITIAL(I); 

DECLARE  (SPLT_VL ,  TMP)  FLOAT  DECIMAL; 

DECLARE  FND  BIT ( 1) ; 

DECLARE  TRUE  BIT(1)  INITIAL  ('  1  '  B)  ; 

DECLARE  FALSF  3IT(1)  INITIAL  (*  O' B)  ; 

CHS_SPLT_VL :  PROCEDURE; 

MPPNT= (UPPF+LWP) /2 ; 

SPLT_VL=LST  (MDPNT)  ; 

LST  (MDPNT)  =LST (LW?)  ; 

FND ; 
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PFP  NG  E_ARND_  SPLT_VL  :  PROCEDURE; 

RTTM=LWR+1 ; 

T  P=  UPP P. ; 

DO  WHILE  ( PTT M<  =  rn P)  ; 

FND=FALSE; 

DO  WHILE  (B!TM<=  TP  P-  ->END)  ; 

IF  LST  (BTTN) <=SPLT_VL  THEM 

prpiTi^  rrg'T'TF  +  1  ‘ 

ELSE 

FND=TPUF ; 

EN  ^ ; 

FND=FAL SE; 

DO  WHILE  (BTTM<=  TP  S  -«FN  D)  ; 

IF  LST  (TP)  >  =  S P LT _ V L  THEN 
I P=TP- 1 ; 

ELSE 

END=TRUE ; 

END; 

IF  FND  THEN 
DC  ; 

IMPREST  (BTTN )  ; 

LST  (BTTN)  =  LS  T  (TP)  ; 
t  ST  (  TP)  =  TUP ; 

TP=TP- 1 ; 

T?\7  ■p  . 

'  -  t 

END; 

S  PLT_T  NDX  =  TP ; 

LST  (LWP)  =  L  S  T  (SPLT_INDX)  ; 

LST (SPLm_INDX) =SPLT_VL; 

FND ; 

SPT_SGMNT ;  PROCEDURE; 

IE  LST  (LWP)  >LST  (UP*)  '“HEM 
DO; 

~H?=LST  (LWP)  ; 

LST (LWP) =  L  ST  (UPP?)  ; 

LST  (UPPR)  =  TF?  ; 

FND; 

FND; 

SGNNT_SZ_GPTF_THN_TW :  PROCEDURE  RETURNS  (3IT(1)  ) 
RETURN  (UPPR-L  WR> 1 )  ; 

END  ; 

EXC7LY_TW_ELMNTS :  PROCEDURE  RETURNS (BIT ( 1) ) ; 
RETURN  ( UPPE-L  WR  = 1 )  ; 

END  ; 
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/*  THE  FOLLOWING  PROCEDURES  MANIPULATE  THE  STACK  OF  SEGMENT 

DECLARE  NMBF._OF_SGMNTS  FIXED  DECIMAL  INITIAL  (1); 

DECLARE  1  STCK  (1:UPPR), 

2  LWR_BNDS  FIXED  DECIMAL , 

2  UPPR_BNDS  FIXED  DECIMAL; 

STCK  __LR  GR_?C  :  PROCEDURE; 

IF  2*SPLT_INDX>UPP  R+LWR.  mREN 
DO; 

LWR_BNDS (NMBR_OF_SGKNTS) =LWR; 

UPPR_BNDS (NMBR_OE_SGMNTS) =SPLT_INDX-  1 ; 

LWF=SPL T_IN  DX+  1 ; 

END; 

ELSE 

DO; 

LWR_BNDS(NMBP_OF_SGMNTS) =SPLT_INDX+1 ; 

UPPR_BNDS (NMBR_OE_SGMNTS) =UPPP ; 

UP  PR=S  PLT_I NDX - 1 ; 

END; 

NMBR_OF_SG  MNTS=NMBR_OF_SGNNTS+ 1 ; 

END  ; 

UNSTCK_NXT_SGMNrn :  PFOCEDURE; 

N  MB  R_C  E_  SG  N  NTS  =  N  M  BP  _0  E_SG  M  NT  S- 1 ; 

IF  STCK  NOT  EMPTY  THEN 
DO; 

LWR=LNR_BNDS (NMBR_OF_SGMNTS)  ; 

UPPB=UPPR_ENDS  (NMBR_OF_SGMNTS)  ; 

END  ; 

FND ; 

STCK_NOT_FMPTY :  PROCEDURE  RETURNS  (BIT ( 1)  )  ; 

RETURN  (NMBR_OF_SG  MNTS>0)  ; 

END; 


*/ 


END  SPT ; 
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£uickersort  (Subroutines,  Meaningless  Variables) 

SOPT:  PROCEDURE  (HIDE , OPTIC) ; 

/*  THIS  PROGRAM  IS  INTENDED  TO  SOPT  AN  ARRAY  */ 

DECLARE  HIDE (*)  FLOAT  DECIMAL; 

DECLARE  OPmIC  ^IXED  DECIMAL; 

DO  WHILE  ( K>TOWN_FAT_PUTT Y)  ; 

IF  GOULASH_MAN  E_SLUMBEP_MILK_EAP  THEN 
DO; 

CALL  THOUGH_MANGE_RADIX ; 

CALL  DEFLATION_DIGEST_MAIZE_OLI VE ; 

CALL  LEFCH_ENTOME_CLOSE; 

END; 

ELSE 

DO; 

IF  SEMINAP_NEW_P.EV ERSAL  THEN 
CALL  HAN  G_DI SSSNT ; 

CALL  HALTING_BILL_CONCERN; 

END; 

END  ; 

/*  THE  FOLLOWING  PPOCEDURES  MANIPULATE  THE  ARRAY  */ 

DECLARE  ( HOARD_TABLE  ,  NOW  ,  MUFFLE,  BE QU  EA.TH)  FIXED  DECIMAL 
DECLARE  HARDY  FIXED  DECIMAL  INITIAL (1); 

DECLARE  (PAISF_FINAL/ POLL)  FLOArp  DECIMAL; 

DECLARE  POACH  BTT(1)  ; 

DECLARE  TRUE  BIT ( 1)  INITIAL  (  *  1  *  B)  ; 

DECLARE  FALSE  BIT(1)  INITIAL  (»  C*  B)  ; 

THOUGH_MANGE_RADIX:  PROCEDURE; 

BE  QUEATH= (OPTIC+ HARDY) /2; 

RAISE_FINAL  =  HIDE (BEQUEATH)  ; 

HIDE  (BEQTTFATK)  =HIDE  (HARDY)  ; 

END; 
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DEFL ATION_DIGEST_MAIZE_OLI VE  :  PROCEDURE ; 

MUFFLE=HAPDY+ 1 ; 

NOW=OPTIC; 

DO  WHILE  (  MUFFLF<=  NOW)  ; 

POACH=FALSE; 

DO  WHILE  (NUEFLE<=NCW  &  -*POACH)  ; 

IF  HIDE  (MUFFLE) <=RAISE_FINAL  THEN 
M!JFFLE=MUFFLE+  1  ; 

ELSE 

POACH = TP  UE ; 

END; 

POACH=EALSF; 

DO  WHILE  (MUFFLE<=NOW  S  -*POACH)  ; 

Ip  HIDE  (NOW)  >=EA.TSE_FINAL  THEN 
NOW= NOW- 1 ; 

ELSE 

POACH =TRUE; 

END; 

IF  POACH  THEN 
DO  ; 

POLL=HIDE (MUFFLE) ; 

HIDE  (MUFFLE)  =HIDE(NOW)  ; 

HIDE  (NOW) =POLL ; 

NOW=NOW- 1 ; 

END; 

END; 

H  OAR  D_  TAB  L  E=N  0  W ; 

HIDE (HARDY) =HTDE ( HOAR D_TABLF) ; 

HIDE  (HOARD_TABLE)  =  RA.I S E_ FINAL ; 

END; 

HAN G_DISSENT :  PROCEDURE; 

IF  HIDE  (HARDY)  >HIDE  (OPTIC)  THEN 
DC; 

POLL=HIDE (HARDY)  ; 

HIDE (HARDY)  =HIDF (OPTIC)  ; 

HIDE  (CPTIC)  =POLL ; 

END; 

END; 

GOULASH_MANE_SLUMBER_MILK_FAR:  PROCEDURE  RETURNS  (BIT  (1)  ) 
RETURN  (OPTIC-^ARDY>1) ; 

END; 

SEMI  NAR_NEW_REVF  RSA.L  :  PROCEDURE  RETURNS  (BIT  ( 1 ))  ; 

RETURN  (OPTIC- HARD Y= 1 )  ; 

END; 
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/*  THE  FOLLOWING  PF.CCEDUFES  MANIPULATE  THE  STACK  OF  SEGMENTS  */ 

DECLAFE  GLOBAL_IN_BF. EFDING  FIXED  DECIMAL  INITIAL  (1); 

DECLAFE  1  DFOOP  (1:OPTIC), 

2  LABOP_GOUFDS  ^IXED  DECIMAL, 

2  NEEDY_ST YLES  FIXED  DECIMAL; 

LEECH_ENTOMB_CLO  SE:  PP.OCEDUEE ; 

IF  2*EOAEP_TABL E>OPTIC  +  HAF  DY  THEN 
DO; 

LABOF_GOUFDS  (GL0BAL_IN_3EEEDIN  G)  =HA?  DY ; 

NEEDY_STYL  ES ( GLOBAL _IN_BE EE DING) =HOAED_T ABLE- 1  ; 
HAEDY=HOAED_TAELE+ 1 ; 

END; 

ELSE 

DO; 

L AEOE_  GOUED  S (G LOBAL_I N_BEEE DIN G) =HOASD_T ABLE* 1  ; 
NEEDY_STYLES  (G LOBAL_IN_BEEEDIN G)  =OPTIC; 

OP  TIC=HOAED_rpABLE-  1  ; 

END  ; 

GLOBAL_IN_BF.EEPING=GLOBAL_IN_BEEEDING  +  1 ; 

END; 

HALTING_BILL_CONCEEN :  PFOCEDUEE; 

GLOBAL_IN_BFEED  ING=GLOBAL_IN_BF. EFDING-  1  ; 

IF  KNOWN_FAT_PU?TY  THEN 
DO; 

H  AF  D Y  =  LAB  OF_GOUEDS  (GLOBAL_IN_BEEEDING)  ; 

OPTIC=NEEDY_ST YLES  (GLOBAL_IN_BFFEDING)  ; 

END; 

END; 

KNOWN_FAT_PUTTY:  PFOCEDUEE  EETUENS  (BIT ( 1 ) )  ; 

FETUFN  (GLOBAL_IN_BEEEDING>0) ; 

END; 

END  SOFT; 
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Q.u±g_k e rsorjt  (Straight-line,  Mnemonic  Variables) 
SOFT:  PROCEDURE  (LIST , UPPER) ; 

/*  THIS  PROGRAM  IS  INTENDED  TO  SORT  AN  ARRAY  */ 


DECLARE 

DECLARE 

DECLARE 

DECLARE 

DECLARE 

DECLARE 

DECLARE 

DECLARE 


DECLARE 

DECLARE 


LIST  (*)  FLOAT  DECIMAL; 

UPPER  FIXED  DECIMAL; 

(SPLIT_INDEX  , TOP , BOTTOM, MIDPOINT)  FIXED  DECIMAL 
LOWER  FIXED  DECIMAL  INITIAL (1); 

(SPLIT_VALUE , TEMP)  FLOAT  DECIMAL; 

FOUND  BIT  (1)  ; 

NU MB ER_OF_ SEGMENTS  FIXED  DECIMAL  INITIAL  (1); 

1  STACK  ( 1 : UPPER )  , 

2  L 0 WE F_ BOUNDS  FIXED  DECIMAL, 

2  UPPEP._ BOUNDS  FIXED  DECIMAL; 

TRUE  BIT ( 1 )  INITIAL ( M ' B) ; 

FALSE  BI T  ( 1 )  INITIAL  (' O' B)  ; 


DO  WHILE  (NUMBFR_OF_SEGMENTS>0) ; 

IF  UPPER-LOWER> 1  THEN 
DO; 

MIDPOINT^  (UPPEF.  +  LOWSP)  /2  ; 
SPLIT_VALUE=LI ST (MIDPOINT) ; 

LIST  (MIDPOINT)  =LIST  (LOWER)  ; 
BOTTOM=LOWER+ 1 ; 

TOP=UPPER ; 

DO  WHILE  (P OTTO M<= TOP) ; 

FOUND=FALSE ; 

DO  WHILE  ( 30  T'TOM<=T OP  &  -.FOUND)  ; 

IF  LIST  (BOTTOM) <=SPLIT_VALUE  THEN 
BOTT  0M= BOTTOM* 1 ; 

El  SE 

FOUND  =  rnRUE ; 

END; 

FOTJND=F ALSF ; 

DO  WHILE  (BOTTOM<=TO?  S  -.FOUND)  ; 

IF  LIST  (TOP)  >=SPLIT_VALUS  THEN 
TOP=  TOP- 1 ; 

EL  SE 

T?OUND  =  TRUE ; 

END; 

IF  FOUND  THEN 
DO  ; 

TEMP=LIST (BOTTOM) ; 

LIST  (BOTTOM)  =LIST  (TOP)  ; 

LIST  (TOP) =TEMP ; 

TOP=TOP- 1 ; 

END  ; 

END; 

SPLIT_INDEX=TOP ; 

LIST (LOWER)  =  LI ST  (SPLIT_INDEX)  ; 

LIST (SPLIT _INP EX) =SPLIT_ VALUE ; 

IF  2*SPLIT  INDEX>UPPER + LOWER  THEN 
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DO; 

LOWER_ BOUNDS (NUM3ER_0F_5EG MENTS) =LOWER; 

U 13 PE R_ ROUNDS  ( NUMBER_OE_S  EG  MENTS )  =  S PL IT_I N  D EX  - 1 
LOWER=SPLIT_INDEX+ 1 ; 

END; 

ELSE 

DO; 

LO WER_POUNDS  ( NUMB ER_OF_ SEGMENTS) =SPLTT_IND EX+ 1 
UPPER_BOUNDS  ( NU  MB  EP._OF_  SEGMENTS)  =UPPER; 
UPPER=SPLIT_TNDEX- 1 ; 

END; 

NUM3EF._0F_SEGMENTS=NUMBER_0F_SEGMENTS+  1  ; 

END; 

ELSE  . 

DO; 

IF  UPPER-LONE R=1  THEN 

DO; 

IF  LIST (LONER)  >LIST  (UPPER)  THEN 

DO; 

TEMP=LIST  (LOWER)  ; 

LIST (LOWER) =LI ST  (UPPER)  ; 

LIST  (UPPER) =TEMP; 

END; 

END; 

NU  MBEE_OF_ SEGMENTS  =  NUMBFR_OE_ SEGMENTS- 1  ; 

IF  NUMBER_OF_S  EGM  ENTS>C  THEN 

DO; 

LO WE?=  LOWF E_BOUNDS  (NTJMBER_OF_  SEGMENTS)  ; 

UPPEP.  =  UPP FR_BOUNDS  (NUMBER_OF_SFG  MENTS)  ; 

END; 

END; 

END  ; 


END  SORT; 
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isor t  (Straight-line,  Shortened  Mnemonic  Variables) 
SRT :  PROCEDURE  (LST,UPPR) ; 


/*  THIS  PROGRAM  IS  INTENDED  TO  SORT  AN  ARRAY  */ 


DECLARE 

DECLARE 

DECLARE 

DECLARE 

DECLARE 

DECLARE 

DECLARE 

DECLARE 


DECLARE 

DECLARE 


LST(*)  FI  OAT  DECIMAL; 

UPPP.  FIXED  DECIMAL; 

(SPLT_INDX,TP,BTTM, MDPNT)  FIXED  DECIMAL; 
LWP  FIXED  DECIMAL  INITIAL (1); 

(SPLT_VL , TMP)  FLOAT  DECIMAL; 

FND  BIT (1)  ; 

NMBF_OF_ SGMNT'S  FIXED  DECIMAL  INITIAL  (1); 
1  STCK  ( 1  :  UPPR)  , 

2  LWF_BNDS  FIXED  DECIMAL , 

2  UPPR_BNDS  FIXED  DECIMAL; 

TRUE  BIT ( 1 )  INITIAL ( ’  1 ' B)  ; 

FALSE  BIT(1)  INITIAL  (’ O' B)  ; 


DO  WHILE  (NMBR_OF_SGMNTS>0 ) ; 

IF  UPPR-LW  P> 1  THEN 
DO; 

MDPNT=  (UPPR  +  LWR) /2; 

SPLT_VL=LST (MDPNT) ; 

LST  (MDPNT)  =LST  (LWR)  ; 

BTTM=L WR+ 1 ; 

TP=UPPR ; 

DO  WHILE  (BT™M<=TP)  ; 

FND  =  FALS  E ; 

DO  WHILE  (BTTM<=TP  F  -FND)  ; 
IF  LST (BTTM) <=SPLT_VL  THEN 
BTTM=BTTM+ 1 ; 

ELSE 

FND=TRTJE  ; 

END  ; 

END=EALSE ; 

DO  WHILE  (BTTM<=TP  &  -FND) ; 
IE  LST  (TP)  >=S?LT_VL  THEN 
T  P=T  P  -  1  ; 

ELSE 

END=rnRUE ; 

END  ; 

IF  FND  THEN 

DO ; 

TMP=L ST (BTTM) ; 

LST (BTTM) =LST  (TP)  ; 

LST (TP) =TMP ; 

TP=TP -  1  ; 


END; 

END; 

5PLT_I NDX=  TP ; 

LST  (LWP.)  =L  ST  (SPLT_INDX)  ; 

LST  (S  PLT_I NDX)  =SPLT_VL; 

IE  2*SPLT  I NDX>UPPR+  LWR  THEN 
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DO  ; 

LWP_BNDS  (NMBE_OF_SGMNTS)  =  LWP; 
UPPE_BNDS  (MMBR_OT?_SGMNTS)  =SPLT 
IP R=SPLT_I NDX+ 1 ; 

END; 

ELSE 

DO; 

LWP_BNDS ( N  MBP_OF_S  GMNTS)  =SPLT_ 
UPPE_BNDS ( NMBR_OF_ SG  MNTS) =  UPPR 
UP PP=S  PLT_INDX - 1 ; 

END; 

NMBR_OF_SGKNTS=NKBR_OF_SGMNTS+1 ; 
END; 

ELSE 

DO; 

IE  UPPP-LW P= 1  THEN 
DO; 

IF  LSI  (LPP) >LST  (UPPF)  THEN 
DO; 

TM P=LS T  (LWR)  ; 

LST  (LWR)  =L  ST  (UPPP.)  ; 

LST  (UP PR) =T  HP ; 

END; 

END  ; 

NKBR_OF_SGM  NTS  =  NNBR_OF_S GM NT S- 1 ; 

IF  NMBF_OF_SGMNTS>C  THEN 
DO; 

LWR=LWR_BNDS (N  MBP_ OF_SGN NT S)  ; 
UPPP=UPPE_ENDS  (NMBP_OE_SGMNTS) 
END; 

END  ; 

END; 


INDX- 1 


INDX+1 ; 

♦ 


♦ 


END  SPT; 
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2ni£kersort  (Straight-line,  Meaningless  Variables) 
SORT:  PROCEDURE  (HIDE , OPTIC) ; 

/*  THIS  PROGRAM  IS  INTENDED  TO  SORT  AN  ARRAY  */ 


DECLARE 

DECLARE 

DECLARE 

DECLARE 

DECLARE 

DECLARE 

DECLARE 

DECLARE 


DECLARE 

DECLARE 


HIDE (*)  FLOAT  DECIMAL; 

OPTIC  FIXED  DECIMAL; 

(HOARD_TABLE , NOW, MUFFLE, BEQUEATH)  FIXED  DECIMAL; 
HARDY  FIXED  DECIMAL  INITIAL (1); 

(RAISE_FINAL, POLL)  FLOAT  DECIMAL; 

POACH  BIT  ( 1)  ; 

GLOBAL_IN_BR  EE DING  FIXED  DECIMAL  INITIAL  (1)  ; 

1  DROOP  (1  :OPTIC)  , 

2  LABOR_GOURDS  FIXED  DECIMAL, 

2  N EEP Y_ST YLES  FIXED  DECIMAL; 

TRUE  BIT ( 1 )  INITIAL (' 1 ' B) ; 

FALSE  BIT  (1)  INITIAL  (»  O' B)  ; 


DO  WHILE  (GLOBAL_IN_BPEEDING>0)  ; 

IF  OPTIC-HAPDY> 1  THEN 
DO; 

BEQUEATH= ( OPTIC+HARDY)  /2  ; 

RAISE_FINAL  =  HIDE (BEQUEATH)  ; 

HIDE  (BEQUEATH)  =HIDE (HARDY)  ; 
MUFFLE=HAKDY+ 1 ; 

NOW=OPTIC; 

DO  WHILE  ( MUFFLE<=NOW)  ; 

POACH=FALSE ; 

DO  WHILE  (MUFFLE<=  NOW  &  -.POACH)  ; 

IF  HIDE (MUFFLE) <=RAI SE_FINAL  THEN 
MUEFLE=M  UFELE+ 1 ; 

ELSE 

POACH =TR  UE ; 

END  ; 

POACH=EALSE ; 

DO  WHILE  ( MU FFLE<=NOW  &  -.POACH)  ; 

IE  HIDE (NOW) >=RATSE_FINAL  THEN 
NOW=NOW- 1 ; 

ELSE 

POACH=TR  UE ; 

END  ; 

IF  POA.CH  THEN 
DO  ; 

POLL  =  HIDE (MUFFLE)  ; 

HIDE (MUFFLE)  =HIDE (NOW)  ; 

HIDE  (NOW) =?OLL ; 

NOW=NO W-  1  ; 

END; 

END; 

HOARD_TABLE=NOW; 

HIDE (HARDY) =HIDE ( HOARD_TABLE) ; 

HIDE (HOARD_TABLE) =? AT S E_ FINAL ; 

IF  2-HOARD  TABLE>OPTIC+HARDY  THEN 
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DO  ; 

LAB 0R_ GOURDS  (GLOBA.L_IN_  BREEDING)  =  HA?  DY 
NEEDY_STYLES (GLOBAL_IN_B REE DING)  =HOARD 
HARDY=HOARD_TABLE+ 1 ; 

END; 

ELSE 

DO; 

LABOR _ GOURDS (GLOBAL_IN_BREEDING) = HOARD 
NEEDY_STYLES  (GLOBAL_IN_BRT’EDING)  =OPTIC 
OPTIC=  HOAR  D_TA.BLE-  1 ; 

END; 

GLOBAT _IN_ FREE DING  =  GLOBAL_IN_BR EE DIN G+ 1  ; 
END; 

ELSE  . 

DO; 

IF  OPTIC-H AP  DY  =  1  THEN 
DO; 

IE  HIDF (HARDY) >HIDE (OPTIC)  THEN 
DO; 

POIL=HIDE (HARDY) ; 

HIDE (HARDY) =  HIDF (OPTIC) ; 

HIDE  (OPTIC) =POLL; 

END; 

END  ; 

GLOBAL_IN_BRi:'EDI  NG  =  GLOBAL_IN_BREE  DING-  1; 

IF  GLOEAL_TN_BP EED IN G> 0  THEN 
DO; 

HARDY=T ABOR_GOURDS  (G LOBAL_I N_BP EEDING)  ; 
OPTIC=N EEDY_ST YLES  (GLOBAL_I N_EP EEDIN G)  ; 
END; 

END  ; 

END; 


TABLE-1 


TABLE+1 


END  SOP^; 
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"WirCli^s^  Eight  Queens  (Subroutines,  Mnemonic  Variables) 

QUEEN S :  PROCEDURE  OPTION S ( MAIN) ; 

/*  THIS  PROGRAM  IS  INTENDED  TO  SOLVE  THE  PROBLEM  OF  PLATING  */ 
/*  EIGHT  QUEENS  ON  A  CHESSBOARD  SUCH  THAT  NO  TWO  QUEENS  LIE  */ 
/*  ON  THE  SAME  ROW,  COLUMN,  OR  DIAGONAL  */ 

DECLARE  (ROW, COLUMN)  FIXED  DECIMAL; 

DECLARE  SAFE  BIT(1); 

DECLARE  TRUE  BIT(1)  INITIAL  ( '  1 ' B)  ; 

DECLARE  FALSE  BIT(I)  INITIAL  ( ’ 0 ' B)  ; 

CALL  CON SIDE  R_FIR  ST_COLUMN ; 

DO  WHILE  (iLAST_COLUMN_DON  E  F  -.RE  GRES  S_OUT_OF_FI R  ST _COLU  MN) 
SAFE=FALSF; 

CALL  TRY_COLUMN ; 

IF  SAFE  THEN 
DO; 

CALL  S FT_QUEEN ; 

CALL  CONST DFR_NEXT_COLUMN ; 

FND ; 

ELSE 

CALL  REGRESS; 

END; 

IF  LAST_COLUMN_DONE  THEN 
CALL  PRI  NT'_ SOLUTION  ; 

T  PY_ COLUMN :  PROCEDURE; 

DO  WHILE  (-.SAFE  &  --LAS T_S QUARE)  ; 

CALL  ADVANCE_POTNTER ; 

CALL  TES  T_S QUARE ; 

END; 

END; 


REGRFSS:  PE  OCEDUF.F ; 

CALL  RECON  SIDEF_PEIOP._COLUMN; 

IF  -<  RE  GRES  S_CUT_0  F_F  IR  ST_COLUMN  THEN 
CALI  REMOVF_QUEEN; 

END; 


-204- 


/*  THE  FOLLOWING  PROCEDURES  HANDLE  THE  BOARD  REPRESENTATION  */ 

DECLAPE  POSITION (9)  ^IXED  DECIMAL; 

CONS  IDE  R_T'IP  ST_COLTJMN  :  PPOCFDUP  E  ; 

COLUMN= 1 ; 

POW=C ; 

END  ; 

CONS IDER_NEXT_COLUMN :  PROCEDURE; 

POSITION (COLUMN) =ROW ; 

COLUMN=COLUMN+ 1 ; 

ROW=C ; 

END  ; 

RECON SIDFP_PFIOF_COLUMN:  PROCEDURE; 

COLUMN  =COL  UMN - 1 ; 

IF  COLU MN>  =  1  THEN 

ROW=POSITION  (COLUMN)  ; 

END; 

ADV  ANCE_POINTEP. :  PRO  CEDUF  E  ; 

RO W=PO  W  + 1  ; 

END; 

P?TNT_SOLUTION:  PROCEDURE; 

PUT  SKIP  LIST  (POSITION  (1)  , POSITION  (2)  , POSITION (3)  , 

POSITION  (4)  , POSITION  (5)  , POSITION  (6)  , POSITION  (7)  , 
POSITION (8) )  ; 

END; 


LAS  T_SQUARE  :  PROCEDURE  RETURNS  (BIT  ( 1 ))  ; 

RETURN  (RO W=8) ; 

END  ; 

LAST_COLU MN_DON'p :  PROCEDURE  RETURNS  (3IT(1)); 

RETURN  (COLUMN>8)  ; 

END  ; 

REGP  E S S_OUT_OF_F IPS T_ COLUMN :  PROCEDURE  RETURNS  (BIT(1)); 
RETURN  (COLUMN<1) ; 

FND  ; 
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/*  THE  FOLLOWING  PROCEDURES  HANDLE  THE  SAFETY  REPRESENTATION  */ 

DECLARE  ROW_SAFE(8)  BIT(1) 

INITIAL  (  (8)  (1)  »  1  ’  B)  ; 

DECLARE  DIAGCNAL_ONE_SAFE (2: 16)  BIT(1) 

INITIAL  (  (15)  (1)  ' 1 *E)  ; 

DECLARE  DIAGONAL_TWO_SA.EE  (-7  :  7)  3IT(1) 

INITIAL  ((15)  (1)  »1'B); 

TES  T_SQUARE :  PROCEDURE; 

S AFE=ROW_S AFE  (FOW)  &  DIAGONAL_ONE_  SAFE  ( ROW  +  COL  TJMS  I 
S  DIAGONAL_TWO_SAFE (ROW-COLUMN)  ; 

END; 

SET  QUEEN;  PEOCEDUFE; 

R  0  W_S  A  F  E ( ROW) =  F  AL  S  E ; 

DIAGONAL_ONE_S  AF  E ( ROW  +  COLU  MN) =FALSE ; 

DIAGONAL_TWO_SAT'E  (ROW-  COLU  MN)  =FA.LSE  ; 

END; 

REND VE_QUEEN;  PROCEDURE; 

ROW_SAFE  (ROW)  =TRUF  ; 

DIAGONAL_ONE_S AFE ( ROW+COLUMN) =TRUE ; 

DIAGONAJj_T  V 0_S  A.F  E  (  ROW  -  COLUMN)  =TRUE  ; 

END; 

END  QUEENS; 
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''WirthJ_s^  Fight  Queen?  {Subroutines,  Shortened  Mnemonic  Variables) 
QNS :  PROCEDURE  OPTIONS ( MAIN)  ; 

/*  THIS  PROGRAM  IS  I NTEN DFD  TO  SOLVE  THE  PROBLEM  OF  PLATING  */ 

/*  EIGHT  QUEENS  ON  A  CHESSBOARD  SUCH  THAT  NO  TWO  QUEENS  LIE  */ 

/*  ON  THE  SAME  ROW,  COLUMN,  OR  DIAGONAL  */ 

DECLARE  (RW , CLMN )  FIXED  DECIMAL; 

DECLARE  S^  BIT ( 1 ) ; 

DECLARE  TRUE  BIT(1)  INITIAL  ( '  1 ' B)  ; 

DECLARE  FALSE  BIT(1)  INITIAL  ( '  0  '  B)  ; 

CALL  CNS D'D_FFST_CLMN  ; 

DO  WHILE  (— . L ST_CL  MN_DN  &  -.RGRS S_ OUT_OF_F RST_CLMN)  ; 

SF=FALSE; 

CALL  TRY_CLMN; 

IF  SF  THEN 
DO; 

CALL  S  FT_QUEEN ; 

CALL  CNSPR_NXT_CLMN; 

END; 

ELSE 

CALL  RGRSS; 

END; 

IF  LST_CLMN_PN  THEN 
CALL  PRNT_  SLTN ; 

TFY_CLMN :  PROCEDURE; 

DO  WHILE  (-.SF  Z  -.LST_SQF)  ; 

CALL  ADVNC_PNTR; 

CALL  T^ST  SQUARE; 

END; 

END  TEY_CLMN; 

RGRSS;  PROCEDURE; 

CALL  RCNSDF._PF  F_CL  MN  ; 

IF  -.RGRS S_OUT_OE_F RST_CL MN  THEN 
CALL  REMOVE_QUEEN; 

END  PGRSS ; 
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/*  THE  FOLLOWING  PROCEDURES  HANDLE  THE  BOARD  REPRESENT  AT  ION  */ 

DECLARE  PSTN  (8)  FIXED  DECIMAL; 

CNS DR_FR ST_CLMN :  PROCEDURE; 

CL  MN= 1 ; 

R  W=C  ; 

END; 

CNS D R_NXT_CL MN :  PROCEDURE; 

PSTN  (CLMN)  =  RW  ; 

CLMN=CLMN+ 1 ; 

R  W  =  0 ; 

END; 

PCNS  DR_PRR_CLMN ;  PROCEDURE; 

CLMN=CLMN- 1 ; 

IF  CLM N>  =  1  THEN 
R W  =  P S T N (CLMN)  ; 

END; 

ADVNC_PNTR :  PROCEDURE; 

R  W  =  F.W  +  1  ; 

END  ; 

PENT  SLTN :  PROCEDURE; 

PUT  SKIP  LIST  (PSTN(1)  , PSTN  (2)  /  PSTN(3)  , 

PSTN  (4)  ,  PSTN  (5)  ,  PSTN  (6)  ,  PSTN  (7)  , 

PSTN  (8)  )  ; 

END; 

LST_SQR :  PROCEDURE  RETURNS  (BIT ( 1 ))  ; 

RETURN  (RW=8) ; 

END; 

LSI_CLMN_DN :  PROCEDURE  RETURNS  (BIT(1)); 

RETURN  (CLMN>8) ; 

END; 

P.GRS  S_OUT_OF_FRS T_CL  MN  ; 

RETURN  (CL  MN< 1 )  ; 

END; 


PROCEDURE  RETURNS  (BI  1(1))  ; 
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/*  m  HE  FOLLOWING  PROCEDURES  HANDLE  THE  SAFETY  REPRESENTATION  */ 

DECLAF2  R  W_SF  (8 )  BIT(1) 

INITIAL  ((8)  (1)  '  1  '  B)  ; 

DECLAEF  PGNL_ON_SF  (2:16)  BIT(1) 

INITIAL  (  ( 15)  (1)  » 1 «B)  ; 

DECLAP  F  DGNL_TW_SF  (-7: 7)  BIT(1) 

INITIAL  (  ( 15)  (1)  •  1  * B)  ; 

TES  T_SQ  UA?  E :  PFOC'EDUFF; 

SF  =  P.W_SF  (F  W)  S  DGNL_ON_SF  (RW+CLMN) 
f  DGNL_TV_SF  (PW-CLMN)  ; 

END; 

SET_  QUFEN :  PROCEDURE ; 

RW_SF (RW) =FALSE; 

DGNL_ON_SF (RW+CLMN) = FALSE; 

DGNL_T W_SF  (RW-CLMN) =FALSE ; 

END; 

REMOVE_QUEEN:  PROCEDURE; 

R  W_SF (RW) =  TRUE ; 

DGNL_CN_SF  (RW+CLMN) =TFUE; 

DGNL_TW_SF (RW-CLMN) =TR  UE ; 

END  ; 

END  QNS; 


^Wirth^s^  Sight  Queens  (Subroutines,  Meaningless  Mnemonic:  Variables) 


QUEENS :  PROCEDURE  OPTION S  (MAIN)  ; 

/*  THIS  PROGRAM  IS  INTENDED  TO  SOLVE  THE  PROBLEM  OF  PLATING  */ 
/*  EIGHT  QUEENS  ON  A  CHESSBOARD  SUCH  THAT  NO  TWO  QUEENS  LIE  */ 
/*  ON  THE  SAME  ROW,  COLUMN,  OR  DIAGONAL  */ 

DECLARE  (PAN, TANDEM)  FIXED  DECIMAL; 

DECLARE  SHOE  BIT(1); 

DECLARE  TRUE  BIT  (  1)  INITIAL  (*  1 '  B)  ; 

DECLARE  FALSE  BIT(1)  INITIAL  ( *  0 1 B)  ; 

CALL  INC LUDED_SMCKE_DI LUTE ; 

DO  WHILE  (->KNOT_GUITAR_WING  &  -«PAISLEY_BUY_TO_GREAI_M  AR3  IN)  ; 
SHOE=FALSE; 

CALL  B A G_ DEACON ; 

IE  SHOE  THEN 
DO; 

CALL  NUT_S PEAR ; 

CALL  LAXAT IVE_OFFER_ROTTEN ; 

END; 

ELSE 

CALL  TERMITE; 

END; 

IF  KNOT_GUITAR_WING  THEN 
CALL  CRT SP_CR EDITOR ; 

BAC_DEACON:  PROCEDURE; 

DO  WHILE  (iSHOF  &  ->HANG_PHRASE)  ; 

CALL  GLARING_EXPFESS ; 

CALL  MAP  E_G R  AN G  E ; 

END; 

END; 

TERMITE;  PROCEDURE; 

CALL  MULTI PLIEf _EE LAY_WINTER ; 

IE  -iPAISLE Y_BU Y_TO_GREAT_MARGIN  THEN 
CALL  SIPHON  LUNGE; 


—  2  1 C  — 


/*  THE  FOLLOWING  PROCEDURES  HANDLE  THE  BOARD  REPRESENTATION  */ 

DECLARE  JAMBOREE  (8)  FIXED  DECIMAL; 

INCL TTDED_SMOKE_D ILUTF  :  PROCEDURE  ; 

TANDEM= 1 ; 

PAN=0 ; 

END; 

LAX ATIVE_OFFER_ROTTEN :  PROCEDURE  ;' 

JAMBOREE (TANDEM) =PAN ; 

TANDEM=TAN  DEM+ 1 ; 

PAN=0 ; 

END; 

MULTIPLIER_RELAY_WINTER:  PROCEDURE; 

TANPEM  =  TAN  DEM- 1 ; 

IF  TANDEM>= 1  THEN 

?AN= JAMBOREE (TANDEM)  ; 

FND ; 

GLAP.ING__EXPP.ESS;  PROCEDURE; 

PAN=PAN+ 1 ; 

END  ; 

CRI SP_CREDITOR :  PROCEDURE; 

PUT  SKI?  LIST  (JAM  EOR  EE ( 1 )  , JAMBOREE (2)  , JAMBOREE (3 )  , 

JAMBOREE  (U)  ,  JAMBOREE  (5)  ,  JAMBOREE  (6)  ,  JAMBOREE  (7)  , 
JAMBORFE  (8) )  ; 

FND; 

HANG_PHRASE:  PROCEDURE  RETURNS  (BIT ( 1 ))  ; 

RETURN  (PAN=8) ; 

FND; 

KNOT _GU I TAR _ WING :  PROCEDURE  RETURNS  (BIT(1)); 

RETURN  (TANDEM>8) ; 

END; 


PAI S  LEY_BUY_TO_GREAT_MARGIN :  PROCEDURE  RETURNS  (BIT(1)); 
RETURN  (TANDEMX  1 )  ; 

END; 
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/*  THE  FOLLOWING  PROCEDURES  HANDLE  THE  SAFETY  REPRESENTATION  */ 

DECLARE  DIG_SIGN  (8 )  BIT  ( 1 ) 

INITIAL  (  (8)  (1)  ’  1  «B)  ; 

DECLARE  UNPOETIC_PUT_CROP (2: 16)  BIT(1) 

INITIAL  (  (15)  (1)  « 1 'B)  ; 

DECLARE  DEBILI TY_M AT_FEPN (-7: 7)  BIT(1) 

INITIAL  (  (15)  (1)  *1  ' B)  ; 

HAH E_ GRAN  GE:  PROCEDURE; 

5HOE=DIG_SIGN (PAN)  &  UNPOET IC_PU T_ CROP (PAN  +  TAN  DSM( 

&  DEBILITY_MAT_FERN  (PAN- TANDEM)  ; 

END  ; 

NUT_  SPEAR ;  PROCFDURF ; 

DIG_SIGN  (PAN)  =FALSE; 

UNPOET IC_ PUT_CPO  P ( PAN+  TANDEM) =FALSE; 

DEBILITY_MAT_FERN  (PAN- TANDEM)  =FA.LSE; 

END; 

SIPHON_LUNGF;  PROCEDURE; 

PIG_SIGN  (PAN)  =TRUE  ; 

UNPOETIC_PUT_CROP (PAN+TANDEM) =T  RUE ; 

DE3ILITY_MAT_F  FRN (PAN- TANDEM) =TRUE ; 

END; 

END  QUEENS; 
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"Wirthis^  Ei^ht  Queens  (S  tra  ight -  line  ,  Mnemonic  Variables) 
QUEENS:  PROCEDURE  OPTION S  (MAIN)  ; 

/*  THIS  PROGRAM  IS  INTENDED  TO  SOLVE  THE  PROBLEM  OF  PLATING  */ 
/*  EIGHT  QUEENS  ON  A  CHESSBOARD  SUCH  THAT  NO  TWO  QUEENS  LIE  */ 


/*  ON  THE  SAME  ROW,  COLUMN,  OR  DIAGONAL 

DECLARE  (ROW, COLUMN)  FIXED  DECIMAL; 

DECLARE  POSITION  (8)  “FIXED  DECIMAL; 

DECLARE  SAFE  BIT(1)  ; 

DECLARE  ROW_ SAFE (8)  3IT(1)  INITIAL  ((8)  (1)  ' 1 » B)  ; 

DECLARE  DIAGONAL_ONE_SAFE (2: 16)  BIT(1)  INITIAL  ((15) 
DECLARE.  DIAGONAL_TWO_SAFE  (-7 :7)  BIT(1)  INITIAL  ((15) 
DECLARE  TRUE  BTT(1)  INITIAL  (»1'B); 

DECLARE  FALSE  BIT(1)  INITIAL  ( •  0  '  B)  ; 

COLUMN=1 ; 

ROW  =  C' ; 

DO  WHILE  (COLUMN>= 1  &  COLUMN<=8) ; 

SAFE=FALSF ; 

DO  WHILE  (-.SAFE  &  ROW<8)  ; 

ROW  =  EOW+  1  ; 

SAFE=  ROW_SAFF (ROW) 

&  D IAG  0  N  AL _0 N  E_S  AF  E  (  R 0  W  +  CO  L  U  M  N ) 

8  DIAGONAL_TWO_SAFE (ROW-COLUMN) ; 


*/ 


0) 

(D 


'  1  '  B) 
'  1  »B) 


END 


'HEN 


IF  SAFE 
DO; 

ROW_SA.FE  (ROW)  =  FA.L  S  E  ; 

DIAGONAL_ONE_S AFE (ROW  + COLUMN)  =FALSE ; 
DIAGONAL_TWO_SAFE (ROW-COLUMN)  =FAL  SE ; 
POSITION (COLUMN) =ROW ; 

C  OLUMN=COL UMN+  1  ; 

RO W=0 ; 

END; 

ELSE 

DO; 

CO  LU MN  =COL UMN -  1  ; 

IF  COLUMN>= 1  THEN 
DO; 

ROW=POSITI ON (COLUMN) ; 

ROW_SAFE (ROW) =TEUE ; 

DIAGON AL_ON  E_SAFE (ROW+COLUMN)  =TRU3 ; 
DI AGON AL_TWO_SAFE (ROW-COLUMN) =TRUB; 
END; 

END; 

END; 

IF  COLU MN>8  THEN 

PUT  SKIP  LIST  (POSITION  (1)  , POSITION  (2)  , 
POSITION (3)  , POSIT ION  (4)  , POSITION (5)  , 
POSITION (6)  , POSITION  (7 )  , POS IT  ION  (  8)  )  ; 
END  QUEENS; 
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"ElElJils"  Eig;ht  Queens  (Straight-line,  Shortened  Mnemonic:  Variables) 
QNS :  PROCEDURE  OPTIONS  (MAIN)  ; 


/*  THIS  PROGRAM  IS  INTENDED  TO  SOLVE  THE  PROBLEM  OF  PLAIING  */ 
/*  FIGHT  QUEENS  ON  A  CHESSBOARD  SUCH  THAT  NO  TWO  QUEENS  LIE  */ 
/*  ON  THE  SAME  FOW,  COLUMN,  OF.  DIAGONAL  */ 


DECLARE  ( RW  , CLMN )  FIXED  DECIMAL; 

DECLARE  PSTN (8)  FIXED  DECIMAL; 

DECLARE  S17  BIT  (1 )  ; 

DFCLARE  RW_SF  (8)  BIT(1)  INITIAL  ((8)  (1)  '  1 ' 3)  ; 

DECLARE  DGNL_ON_SF  (2  : 16)  BIT(1)  INITIAL  ((15)  (1)  '  1  '  B)  ; 
DECLARE  DGNI  _TW_ST’  (- 7 :  7)  BIT(1)  INITIAL  ((15)  (1)  '  1  '  B)  ; 
DECLARE  TRUE  BIT  ( 1)  INITIAL  ( *  1 '  B)  ; 

DECLARE  FALSE  3IT(1)  INITIAL  ( 1 C ' B)  ; 


CLMN=  1 ; 

R W=0  ; 

DO  WHILE  (CL MN>= 1  &  CLMN<=8)  ; 
SF=FALSE; 

DO  WHILE  (--SF  8  RW<8)  ; 

R W=F  W+ 1 ; 

SF=P.W_SF  (RW) 

S  DGNL_ON_SF  (RW+CLMN) 

F  DGNL_TW_SF  (RW-CLMN)  ; 

FND ; 

IF  SF  THEN 
DO; 

RW_SF  (RW)  -  "^A.LS  E  ; 

DGNL_ON_SF  (RW+CLMN)  =FALS|; 
DGNL_T W_SV  (RW-CLMN)  =FA.LSE; 
PSTN (CLMN) =  RW  ; 

CLMN=CLMN+  1  ; 

RW  =  0  ; 

END; 

ELSE 

DO; 

CL  MN=CLMN- 1 ; 

IF  CLM N>= 1  THEN 
DO; 

RW =PSTN (CLMN)  ; 

RW  _SF (RW) =  TRUE ; 

DGNL_ON_SF  (RW  +  CLMN) =TRUE ; 
DGNL_T W_SF  (RW-CLMN) =TRUE ; 
FND; 

END  ; 

END; 

IV  CL MN>  8  THEN 

PUT  SKIP  LIST  (PSTN  (1)  ,PSTN  (2) 
PSTN  (3)  ,PSTN  (4)  ,PSTN  (5)  , 

PSTN  (6  )  ,  PS  TN  (7)  ,PSTN(8))  ; 

FND  QNS; 


/ 
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Eig.ht  (Straight-line,  Meaningless  Variables) 

QUEENS:  PROCEDURE  OPTION  S  (MAIN)  ; 

/*  THIS  PROGRAM  IS  INTENDED  TO  SOLVE  THE  PROBLEM  OF  PLATING  */ 

/*  EIGHT  QUEENS  ON  A  CHESSBOARD  SUCH  THAT  NO  TWO  QUEENS  LIE  */ 

/*  ON  THE  SAME  ROW,  COLUMN,  OR  DIAGONAL  */ 

DECLARE  (PAN, TANDEM)  FIXED  DECIMAL; 

DECLARE  JAMBOPEE (8)  FIXED  DECIMAL; 

DECLARE  SHOE  BIT (1) ; 

DECLARE  DIG_S IGN ( 8)  BIT(1)  INITIAL  ((8)  (1)  *1*3) ; 

DECLARE  UNPOETIC_PUT_CPO?  (2:  16)  BIT(1)  INITIAL  ((15)  (1)  M'B) 
DECLARE  DEBILITY_MAT_FERN  (-7: 7)  EIT(1)  INITIAL  ((15)  (1)  M'B) 
DECLARE  TPUE  BIT(1)  INITIAL  (M'B); 

DECLARE  FALSE  3Im(1)  INITIAL  ( '  0 ' 3)  ; 

TANDEM= 1 ; 

P  A  N  =  0  ; 

DO  WHILE  (TANDEM>=1  8  TANDEM <=  8)  ; 

SHOE=F ALSE ; 

DO  WHILE  (-.SHOE  8  PAN<8)  ; 

PAN  =  PAN+ 1  ; 

SHOE=DIG_SIGN  (PAN) 

8  UNPOETIC_PUT_CRC? (PAN+TANDEM) 

8  DEBILITY_MAT_EERN (PAN-TANDEM) ; 

END; 

IE  SHOE  THEN 
DO; 

DTG_ST  GN (PAN) =EALS  E; 

UN  POET  IC_PUm_CP.O?  (PAN  +  TANDEM)  =EALSE  ; 

DEBILITY_MA.T_FERN  (PAN-TANDEM)  =EAL SE  ; 

JAMBOPEE (^ANDEM) =  PAN ; 

T  AN  D  E  M = TAN  D  E  M  +  1  ; 

PAN=0; 

END ; 

ELSE 

DO; 

T AN  D  E  M  =  TAN  DEM-  1  ; 

IF  TANDEM>= 1  THEN 
DO; 

PAN=JAM30REF (TANDEM)  ; 

DIG_SIGN  (PAN)  =TRU  E  ; 

UN POErr  IC_?UT_CFO'°  (PAN+TANDEM)  =TRUE  ; 

DEBILIT Y_MAT_FERN (PAN- TANDEM) =T  RUE ; 

END ; 

END; 

END  ; 

IF  TANDEM>8  THEN 

PUT  SKIP  LIST  (JAMBOREE (1)  , JAMBOREE (2)  , 

JAMBOPBE(R)  , JAMBOREE (4)  , JAMBOREE (5)  , 

JAM30F EE (6)  , JAMBOREE (7)  , JAMBOPEE (8)  )  ; 

END  QUEENS; 
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T-.ppendix  M 


SAMPLE  INSTRUCTION  BOOKLET  USED  IN  EXPERIMENT  10 
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Name 


I nstr uct ions 


This  experiment  consists  of  various  sections.  You  will 
be  given  a  fixed  amount  of  time  for  each  of  the  sections. 
Do  not  at  any  time  go  back  to  previous  sections  and  only 
move  ahead  when  instructed  to  do  so.  The  various  sections 
and  the  time  limits  are  as  follows: 


reading  the  program 

5 

m  i  n  ii  t  e  s 

detailed  study 

15 

minut es 

quiz 

10 

minutes 

modifying  the  program 

10 

minutes 

quiz  and  questionnaire 

5 

minutes 

hr oughout 
Accuracy 

this  experiment,  emphasize 
is  worth  80%,  speed  20%. 

accuracy  over 

When  instructed  to  do  so,  read  the  program  which  you 
have  been  given  until  time  is  called.  Do  noji  attempt  to 
hand- simulate  it.  Go  on  to  the  next  page  only  when 
instructed  to  do  so. 
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Circle  a  number  from  0  to 
feel  you  understand  the  program.  0 
means  "I  can  explain  what  it  does", 
why  it  works",  9  means  MI  underst 
other  numbers  indicate  intermedia 


9 

ind 

1 

ca 

ting  how 

well 

you 

mean 

c; 

ii 

not  at 

all"  , 

3 

ti 

,  6 

m 

ea 

ns  "I  c a 

n  explain 

d 

it 

p 

er 

f  ect ly" , 

and 

l.  h  0 

stat 

e 

s 

between 

these 

f 


1 


7 


8 


Study  the 
recommended  than 


program  using  any  method  you  wish  It  is 
you  attempt  to  h and- simulate  the  program. 


Go  on 


.he  nex  +  page  only  when  instructed  to  do  so 
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Circle  a  number  from  0  to  9  indicating  how  well  you 
feel  you  understand  the  program.  0  means  "not  at  all",  3 
means  "I  can  explain  what  it  does",  6  means  "I  can  explain 
why  it  works",  9  means  "I  understand  it  perfectly",  and  the 
other  numbers  indicate  intermediate  states  between  these., 

C  123456789 

Answer  the  following  10  questions  about  the  program 
You  may  refer  back  to  the  program  as  you  are  attempting  to 
answer  the  questions. 

1.  After  choosing  a  particular  element,  how  is  an  array 
segment  partitioned  using  this  element? 


2.  Consequently,  where  is  this  element  placed  in  the  array? 


3.  The  section  of  the  program  which  partitions  an  array 
segment  consists  of  an  outer  DO -WHILE  loop  which  contains 
two  inner  DO-WHILE  loops.  What  does  the  first  inner  loop 
d  o? 


4,  What  does  the  second  inner  loop  do? 


5.  What  does  the  outer  loop  do? 


6,  What  is  the  purpose  of  statement  35? 


7,  When  does  the  program  decide  not  to  further  partition  a 
segment? 


8, 


What  does  it  dc  in  this  case? 


Whan  is  the  purpose  of  statement  65  and  why  is  it  valid 


1?,  Whan  is  the  stack  used  for? 


Go  on  to  +he  next  page  only  when  instructed  to  do  so. 


-2  20- 


Make  the  following  2  modifications  to  the  program.  The 
modifications  are  independent  of  one  another,  and  each  one 
refers  to  the  original  program. 

1,  If  lines  89  -  91  of  the  program  were  replaced  by  the 
f  olio  wing : 

DECLARE  STACK  ( 1 : UPPER,  1 : 2)  EIXED  DECIMAL; 

what  other  changes  would  have  to  be  made  to  the  program? 
(Indicate  the  actual  changes  you  would  make  by  referring  to 
the  line  numbers  of  the  program.) 


2.  If  instead  of  directly  sorting  only  segments  of  size  2, 
we  also  want  to  directly  sort  segments  of  size  3,  which 
lines  would  have  to  be  changed?  (Indicate  the  line  numbers 
in  the  space  below;  you  need  not  indicate  the  actual 
changes. ) 


Go  on  to  the  next  page  only  when  instructed  to  do  so. 


pj  l* 
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Circle  a  number  from  C  to  9  indicating  how  well  you 
feel  you  understand  the  program.  0  means  "not  at  all",  3 
means  "I  can  explain  what  it  does",  6  means  "I  can  explain 
why  it  works",  9  means  "I  understand  it  perfectly",  and  the 
other  numbers  indicate  intermediate  states  between  these,. 

C  123  456789 

Answer  the  following  10  guestions  about  the  program. 
These  are  the  same  guestions  that  you  were  asked  before.  If 
your  answer  is  the  same  as  before  you  may  answer  "same",. 

1.  After  choosing  a  particular  element,  how  is  an  array 
segment  partitioned  using  this  element? 


2,  Consequently,  where  is  this  element  placed  in  the  array? 


3.  The  section  of  the  program  which  partitions  an  array 
segment  consists  of  an  outer  DO-WFTLE  loop  which  contains 
wo  inner  ro-WKTLE  loops.  What  does  the  first  inner  loop 
o? 


U.  What  does  the  second  inner  loop  do? 


5.  What  does  the  outer  loop  do? 


6,  What  is  the  purpose  of  statement  35? 


7,  when  does  the  program  decide  not 
segment? 


to  further  partition  a 
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R,  What  does  it  do  in  this  case? 


What  is  the  purpose  of  statement  65  and  why  is  it  valid? 


1C.  What  is  the  stack  used  for? 


Go  on  to  the  next  page. 


Answer  the  following  questions: 


1.  For  each  of  the  following  factors,  please  circle  a 
number  from  C  to  Q  indicating  the  factor's  contribution  to 
your  understanding  of  the  program  (  C  means  "nor  helpful  at 
all",  9  means  "extremely  helpful"), 

comments  in  program 


C  123456"?«9 

placement  of  declarations 

C  123456789 


paragraphing  of  program 

G  123456739 

choice  of  variable  names 

0  123456789 


choice  of  control  structures 


C 


3  4  5 


6  7  8  9 


2,  Do  you  recognize  this  program  as  similar  to  one  rhar  you 
have  seen  before?  (Please  explain  when  and  how  carefully 
you  srudied  it  then,) 


3.  Did  you  have  any  trouble  understanding  any  of  the  Pl/i 
constructs  used  in  this  program?  Which  ones? 


4.  Can  you  thir.h  of  any  other  reason  why  your  performance 
on  this  experiment  may  not  be  typical? 
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5,  Please  add  any  additional  comments  on  the  program  or  the 

experiment : 
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